Sure looks as if Holmes is being rude, Becky. I watched that bit 3 or 4 times and while there's no lip movement, who else could it be? He's the one who walked away...alone, after addressing 'gentlemen', plural.
Glad Watson is still Watson. Though who knows, they may send him off to darkest Africa in the next one and give murderous Holmes a sidekick like Mrs. Peel. Ah, those tight motorcycle leathers!