And ye shall take you on the first day the boughs of goodly trees, branches of palm trees, and the boughs of thick trees, and willows of the brook; and ye shall rejoice before the LORD your God seven days (Leviticus 23:40).
This is a commandment that the rabbis felt needed to be observed. The key phrase here is “the fruit of goodly trees.”
They interpreted this phrase as meaning both the tree and the fruit had to taste goodly. They felt the fruit and the tree needed to taste similar. They claimed the etrog was the only fruit that met the requirement.
Adding weight to this matter is the mathematical value (using Gematria) of the Hebrew word etrogim (plural of 1 etrog) was the same as the mathematical value of the Hebrew phrase “fruit of a goodly tree.”
Since the numerical value was the same they decided that this was the fruit.
There are some rabbis who hold to a different interpretation (Sukkah 35a).
Read more about the holiday of The Feast Of Tabernacles.
Articles of interest include:
From The Fall Feast of Israel by Mitch and Zhava Glaser.