The Tora text states that Yosef’s brothers hated him both for
a) the ways in which Yosef acted towards them—he took every opportunity to cast them in a negative light in their father’s eyes:
Beraishit 37:2
…and Yosef brought evil reports about them to their father,
and how he disclosed to them his dreams that implied that Yosef was more important than they were in the grand scheme of things:
Ibid 7
Behold we were gathering sheaves in the field and behold my sheaf stood up and remained erect, and your sheaves surrounded it and bowed down to it.
Ibid. 9
…Behold the sun and the moon and eleven stars were bowing down to me.