It has been narrated on the authority of Said bin Musayyab who said: My father was one of those who swore fealty to the Messenger of Allah ﷺ near the tree. When we passed that way next year intending to perform the Hajj, the place of the tree was hidden to us. If you could point out clearly, you would (certainly) be knowing better.
It has also been narrated on the authority of Said bin Musayyib who learnt from his father that they were with the Messenger of Allah ﷺ in the year of the Tree (i. e. in the year of the fealty of Gods pleasure sworn under the tree at Hudaibiya), but next year they forgot the spot of the tree.