This sentence, I've always thought, was rather unsettling and ugly, as were the words written to Wilson you quote. Nabokov simply could not transcend his bigoted feelings about his brother's sexuality and so his tributes are cutting and condescending at the same time as they try to express regret. MIchael Wood in his discussion of Speak Memory makes much of these lines, and Nabokov's unworked out feelings of guilt over Sergei.

To be fair to Nabokov though, Stacy Schiff revealed in her biography of Vera that Sergei probably looked down on N's marriage to Vera because she was a Jew and that he (Sergei) and Vera never got on; Sergei wished N had married his first fiance, Svetlana Siewert (I think that's right) a devout catholic whose family made her break with him because his money prospects, as a struggling writer, were so poor (which influenced the the portrayal of the family of Luzhin's fiance in The Defense). In fact, I believe that Boyd says that Nabokov had at first jumped to the conclusion that Sergei had collaborated with the Nazis and was getting ready to denounce him as such when he discovered Sergei had spoken out against Hitler at his place of employment, been informed upon, arrested, and then died. This would have been a lot for Nabokov to come to terms with, even if he had been given to airing his dirty laundry, so the vague congested tone of the passage is hardly surprising.