Here is a doctor who recommends using a family history of peanut allergy. See Group 4 here:
Group 4: No/mild history of eczema or food allergy but a family history of peanut allergy
Data regarding this group are very limited
Due to potential benefit, it is recommended that these infants have age appropriate peanut containing foods introduced into their diets around 4-6 months of age
However, due to a lack of studies in this cohort, evaluation and method of introduction can be left to the discretion of parent and health care provider preference
Nothing there about testing for egg allergy.
So, you are right that some doctors may use family history at their discretion.