Video games provide an interesting framework of rules, actions, events, and user interaction for the exploration of music expression. In this paper we describe a set of computer games designed for group improvisation and controlled by playing musical instruments. The main contribution of our work is the two-way interaction between music and video games, as opposed to the more commonly explored one-way interaction. We investigate the different challenges involved, such as finding adequate game controlling events, provide enough expressive freedom to musicians, correct playing speed and game complexity, and different artistic expression forms. We also present the problems encountered, design considerations, and different proposed and tested solutions. The games developed in this project were used in a concert and a set of workshops.