Introduction of diseases may have been inevitable ("spread and infect" is what a successful pathogen does, after all), but the genocides that occurred int the Americas certainly were not inevitable; they were man-made atrocities.
Consistently, European colonizers utilized the fracturing and weakening of American societies to exploit and exterminate groups. The idea that the depopulation of the Americas was an inevitable function of the introduction of Afro-Eurasian diseases ignores the fact that even after the initial epidemics wiped out a tremendous portion of the population, the numbers of Americans not only failed to recover, but continued to decline. Attributing this continued decline to epidemic disease elides over the fact that, in the wake of these tragedies, Americans were exploited and abused, or exterminated when they resisted.
We can see how European actions, outside of disease processes, led to massive depopulation basically from the moment colonizers arrived. Bartolome de las Casas published his Brief Account of the Destruction of Indies in 1552, where he described the depopulation of Hispaniola:
[The Spanish] sent the Males to the Mines to dig and bring away the Gold, which is an intollerable labor; but the Women they made use of to Manure and Till the ground, which is a toil most irksome even to Men of the strongest and most robust constitutions, allowing them no other food but Herbage, and such kind of unsubstantial nutriment, so that the Nursing Womens Milk was exsiccated and so dryed up, that the young Infants lately brought forth, all perished, and females being separated from and debarred cohabitation with Men, there was no Prolification or raising up issue among them. The Men died in Mines, hunger starved and oppressed with labor, and the Women perished in the Fields, harrassed and broken with the like Evils and Calamities: Thus an infinite number of Inhabitants that formerly peopled this Island were exterminated and dwindled away to nothing by such Consumptions.
Disease was not necessary. Forced labor, which broke the body but also broke the people through starvation and deprivation, ground a people already devastated by disease in to extinction; the Taino had ceased to exist as a people before the Mayflower landed.
When Europeans did arrive in the continental North America, they similarly exploited the situation of Americans weakened by disease, though often the goal was direct removal of the latter group from the picture, often using resistance to colonization as the justification. The Pequot War in 1637 began when an English punitive expedition burned two Pequot villages, in justification for the raiding and death of an Englishman that the Pequot had nothing to do with. That war famously ended with the Mystic Massacre, in which several hundred women, children, and old men in an undefended village were burned alive, or killed as they tried to flee the flames. The remaining Pequot were either sold into slavery or absorbed into other American groups.
Similarly, the 1711 Tuscarora War was sparked by raids, murders, slaving, and land-taking by the English in Virginia. When the Tuscarora retaliated, the House of Burgesses, citing the precedent set by another colony, declared:
it is resolved that no peace, or any terms whatsoever be agreed with them but entirely to extirpate them according to the laudable custom of South Carolina.
David Stannard, in his seminal work, American Holocaust: The Conquest of the New World, notes an abundance of instances where American groups lost population, often to the point of extinction or near extinction, even outside of disease. He notes that even during WWI, the English population continued to increase. As did the Japanese population during WW2. The total war against the Americans by European colonialists consistently resulted in the very foundation of survival being eliminated. He writes:
The European habit of indiscriminately killing women and children when engaged in hostilities with the natives of the Americas was more than an atrocity. It was flatly and intentionally genocide. For no can survive it its women and children are destroyed
The Indian removals which occurred under the US government were likewise not "inevitable." They were a deliberate choice by the US government to privilege the desires of Euro-Americans over the rights of Native Americans. That these removals occurred with the promise of compensation (often inadequate or undelivered compensation) does not make them any less destructive. Purchasing land as a more effective way of driving indigenous groups ever westward was recognized by George "Town Destroyer" Washington himself. In his 1783 letter to James Duane, he wrote:
I am clear in my opinion, that policy and economy point very strongly to the expediency of being upon good terms with the Indians, and the propriety of purchasing their Lands in preference to attempting to drive them by force of arms out of their Country; which as we have already experienced is like driving the Wild Beasts of the Forest which will return as soon as the pursuit is at an end and fall perhaps on those that are left there; when the gradual extension of our Settlements will as certainly cause the Savage as the Wolf to retire; both being beasts of prey tho’ they differ in shape. In a word there is nothing to be obtained by an Indian War but the Soil they live on and this can be had by purchase at less expense, and without that bloodshed, and those distresses which helpless Women and Children are made partakers of in all kinds of disputes with them.
Washington's more moderate, though profoundly paternalistic, ideas were still predicated on the idea that Native Americans must give way to European colonizers. Moreover, his proscriptions that the dealings be fair and mutual were obviously not followed. The forced deportation and deaths of thousands along the Trail of Tears -- in violation of a Supreme Court ruling -- was certainly not in line with Washington's idea "for every one to obtain what is reasonable and proper for himself upon legal and constitutional ground," and it was not inevitable; it was a deliberate choice.
An even clearer, and bloodier, example of how the choices of Euro-Americans to dehumanize Native Americans led to genocides can be seen in mid-19th Century California. Between mid 1840s and 1870, the indigenous population of that state was reduced from an estimated 120,000 to less than 30,000. This is after the population had already been decimated by introduced diseases and the Spanish mission system. Brendan Lindsay's (2012) Murder State: California's Native American Genocide, 1846-1873 documents how it was both unofficial practice and official policy to disregard or outright encourage the murder of Native Americans and kidnapping of their children. This included local bounties on Indian scalps and heads (50¢ and $5, respectively, in one county) and well as a rampant slave trade. Regarding one group's actions, Lindsay writes:
Two Americans, the Asbill brothers, kidnapped Indian boys and girls; they forced the boys to hunt game as part of a hide-tanning business they had established, and they traded the girls for horses... In one transaction alone the men traded 35 Yuki girls for 105 horses... Euro-Americans used the reservations as markets to pick up Native women and children. These reservations included the Nome Cult Farm and Nome Lackee reservation, where many of the Yuki people had been taken against their will or convinced to relocate for protection from white settlers. The relocations only made it easier for whites to kidnap them, essentially concentrating the targets of the slavers in one convenient spot. Conditions were worse in mining areas where Euro-Americans vastly outnumbered Native populations. Miners kidnapped Native women and drove away or killed Native men as they appropriated everything and everyone they desired by weight of numbers. What they did not take they burned to keep Indigenous populations from coming back.
The brutality inflicted upon the native populations could not, and was not, reigned in by government forces, instead the abuses were codified. The 1850 Act for the Government and Protection of Indians established that "in no case shall a white man be convicted of any offense upon the testimony of an Indian or Indians," which included parents testifying that a kidnapped child was theirs. The taking of native children by whites was actually set forth in the law, as was selling into slavery any adult native found "trolling, loitering where alcohol was sold, begging, or leading a profligate course of life."
The impact of infectious disease may have set the stage for interactions between Europeans and Americans, but the genocides that occurred again and again were deliberate acts. They were often carried out with the precise intention of wiping the native population off the land, with language that portrayed the natives as little more than wild animals. The dehumanization and idea that the colonizers avarice outweighed the humanity of the natives led again and again to disproportionate force and massacres; deals made with with the implicit threat of violence; and the dislocation of groups to unsuitable lands unable to sustain the population. None of this was inevitable.