The answer to this question depends on how one understands the word moladeti in 24:4 and 24:7. Rashi (on 24:7, also Radak and Ibn Ezra on 24:4) explains that moladeti refers to a person’s birthplace which he explains means Ur Kasdim, as that was where he believes Avraham was born. This implies that in 24:4, Avraham was telling his servant to go both to Haran (Avraham’s land) and to Ur Kasdim (Avraham’s birthplace) to find a wife for Yitzhak. However, once the servant was successful in Haran, he did not have to go onto to Ur Kasdim.
Cassuto (1964, pp. 274,275, see also Ramban and Rashbam on 24:7) argues that the word moladeti means extended family and he explains that Avraham was sending his servant to the land of his family, which was Haran. According to him, 24:4 and 24:7 would mean the land of my kinsman, (a hendiadys), see Sarna 1989, p. 162. Among the proofs that Cassuto brings is that Bereshit 43:7, 48:6 and Megillat Esther 8:6 (also see Vayikra 18:11) show that the word moledet means family and not birthplace.
N. Leibowitz (1976, pp. 214-222) argues that Rashi’s interpretation of the word moladeti, birthplace and not family, must be correct based on the behavior of the servant. The servant devised a test to choose a wife, and this test was apparently for all the women of Haran. How could the servant have used such a test, if the wife had to be from Avraham’s family? Accordingly, N. Leibowitz argues that it was a coincidence that Rivka was from Yitzhak’s family, and she reviews several reasons why the women of Haran were more suitable for Yitzhak than the women of Canaan. Yet, if really the servant could have chosen anybody, then the choice of a family member seems too coincidental. Furthermore, Yitzhak later would tell Yaakov to marry from his family (28:2) and if it was just coincidence that Yitzhak married within the family, why was Yaakov told to marry within the family? Finally, 22:20-24 tells us of the birth of Rivka and that this information was told to Avraham. If Avraham was not concerned if Yitzhak married within the family, why did the Torah have to tell us that Avraham learned of Rivka’s birth? (See Seforno on 22:23, I thank David Barrett for this source.)
As I discuss on Bereshit 24:12-61 "All the world's a stage," I believe that the test devised by the servant was not a random test. The servant had already chosen Rivka and the test was staged to get the family’s agreement to the match. The servant ran to Rivka, he gave her presents before he asked her name, and he referred to her as Rivka without ever being told her name. The servant’s test was not open to all the women of Haran, and the test is no proof that Avraham was willing for his son to marry anybody from Haran. Thus, the word moledet means extended family, as argued by Cassuto. The servant was not lying when he recounted Avraham's instructions, but his use of the word mishpacha, did imply a smaller circle of family members than Avraham had told him.
Why did Avraham want Yitzhak’s wife to be from his family? Yehoshua in his final address to the people reviewed their history and he stated that Terah and Nahor, Avraham’s family, were idol worshippers, Yehoshua 24:2, what made a member of Avraham’s extended family more suitable for Yitzhak than anybody else? My guess is that relatively the people in Avraham’s extended family were less idol worshippers than other people.
There are two proofs for this assertion. One, the whole family of Terah was part of the initial journey from Ur. 11:31 even records that Terah was the initiator of this trip. This journey was not simply a normal migration, but rather the beginning of the mission of Avraham. Therefore, it is likely that the whole family and not just Avraham had some notion of G-d, see our discussion on Devarim 32:10, "Who did G-d find and protect in the desert?"
Secondly, when the servant came to speak to the family, Lavan said, “Come in, O blessed of G-d” (24:31) and after the servant finished speaking, Lavan and Betuel stated “the matter came from G-d” (24:50) both times using the personal name for G-d. Presumably, only one who has some idea of G-d would make such statements. This relative belief in G-d explains why Yitzhak wanted Yaakov to marry within the family. However, over time these religious ideas and feelings started to wane, as evidenced by 31:53, where Lavan distinguished between the G-d of Avraham and the G-d of Nahor. Accordingly, we do not know that any of Yaakov's children went back to the family of Nahor to find wives.
Bibliography:
Cassuto, Umberto (1883-1951), 1964, A commentary on the book of Genesis, part two: From Noah to Abraham, Jerusalem: Magnes Press.
Leibowitz, Nehama (1905-1997), 1976, Studies in Bereshit, translated by Aryeh Newman, Jerusalem: The World Zionist Organization.
Cassuto, Umberto (1883-1951), 1964, A commentary on the book of Genesis, part two: From Noah to Abraham, Jerusalem: Magnes Press.
Leibowitz, Nehama (1905-1997), 1976, Studies in Bereshit, translated by Aryeh Newman, Jerusalem: The World Zionist Organization.