When the government imposes tariffs or import quotas, it harms a few foreigners — exporting producers and their workers mainly — but it harms far more people in the country with these trade obstructions, who suffer an absence of superior options or face higher prices for the imported types of goods on the domestic market owing to lessened competition.
Likewise, when the government imposes restrictions or quotas on immigration, it harms a few foreigners — the foreigners who wish to enter the country mainly — but it harms far more people in the country with these obstructions, who suffer an absence of superior options in labor and other markets or face higher prices for the immigrants’ types of services, besides being deprived of their freedom to deal with their most preferred or rewarding trading partners as customers, tenants, neighbors, and friends. Not to mention that enforcement of immigration restrictions fosters the maintenance of a domestic police state that menaces everyone living in the country to which immigrants wish to come.
These two cases of government obstructions at the border thus have many parallels. The important point is that they give rise to harms imposed on the multitude for the sake of creating unearned income for a minority in the country whose government imposes the obstructions.