In the case of Rwanda, many of the weapons were handheld tools such as machetes, knives, et cetera.
The buildup of arms, ammunition and weaponry began in 1990. The country, which was one of the poorest in Africa, spent over $12 million USD on arms, draining funds from it's own economy. As a result, it was one of the biggest importers of arms in Africa, even when compared to relatively affluent nations such as South Africa. In 1992, Rwanda signed a $6 million deal with Egypt, which included primarily explosive armanents, such as land mines, artillery, howitzers, mortars, rocket-propelled grenades, and over three million rounds of small arms ammunition.
In 1993, a French arms firm sold $12 million USD worth of arms to Rwanda. The deal was finalized in 1994. Despite the Arusha Accords (an agreement between the Rwandan Government and the Rwandan Patriotic Front that aimed to stop violence between the two parties) basically prohibiting the mass importation and distribution of weapons, the deal went through, and a French cargo plane secretly deposed the weapons in Kigali, Rwanda.
There are, of course, other ways that the weapons were acquired. Some of the weapons were remnants from Belgium's colonization of the nation, and some of the weapons simply existed within the country, for whatever reason... black market sales, smuggling ,etc.
With all of these weapons within the nation, you could be forgiven for thinking that the United Nations should have been able to seize and destroy the stockpiles that were being acquired and stockpiled en masse by the militias. Well, that isn't exactly true. The commander of the UNAMIR (United Nations Assistance Mission for Rwanda), Lieutenant-General Roméo Dallaire, wanted to try and seize the weaponry to prevent the militias from starting an organized conflict. He had been informed by "Jean-Pierre", a militia commander and former member, that a mass extermination of Tutsi individuals was imminent. Realizing the clear and apparent threat, Dallaire contacted General Maurice Baril, who was a higher-up in the Department of Peacekeeping Operations, asking for approval to seize and destroy the weapons.
He was later informed that he should not do so. The reason? It was not in the mandate of UN peacekeepers to do so. This decision was later seen as a deciding factor in the genocide, with many critics of the United Nations peacekeeping force stating that, with prompt and timely destruction of the weapons, a genocide situation could have been prevented.
To sum up: Rwandan militias and government forces alike collected weapons years before the genocide, mainly through arms agreements, and the UN was unable to destroy them due to a limited mandate.