you need a manual grout removal tool. These typically come in one of two types: One, like the REETREE grout removal tool, ...

These gadgets feature pinpoint heads to get to the grout without damaging the surrounding tile. There are two kinds of remover: an oscillating electric device for fast buffing off the discoloured top ...