There's always some music and song in the Bricklayer's Arms. If somebody tells you the Brickies has closed, it's not true - there was a brief closure but it's re-opened. The brickies usually has songs in one room and tunes in another, but it's all a bit rowdy - fine if you like that sort of thing.
There sometimes a Saturday lunchtime English music session at the New Crown, but the locals take it over in the evening.
There's usually an Irish music session somewhere. (Letter 'B'? Falcon? in that neighbourhood anyway)
I've heard that The George (big new Wetherspoons pub in the market square) allows music, despite being contrary to the chain's usual policy.
Finding a good session can be hard. Definitely worth bringing a box with you, and you can always join in with the band playing for the Bear in the procession. If you don't know the Straw Bear tune, you soon will!
If you bump into Pig Dyke Molly (all black and white) - come and say hello - I'm the older melodeon player.