Possible Clues:

• Tree found in part of London
• Tree — area of London
• Tall tree of north temperate regions
• Tree liked by many, excluding you reportedly
• Softwood tree with catkins
• Tree non-U, if non-U
• Tree — London area
• East London area
• Tree with leaves that quiver
• Tree liked by one's associates -- not yew, by the sound of it