A base is any substance that reacts with an acid to form a salt and water only. Most bases are metal oxides or metal hydroxides, for example copper oxide, magnesium oxide and sodium hydroxide. When a base reacts with an acid the acid is used up, which is why the reaction is called neutralisation.
For example, copper oxide reacts with hydrochloric acid to form copper chloride and water. Copper chloride is a salt. The only products are a salt and water, so copper oxide is a base.
Many bases, such as copper oxide and magnesium oxide, do not dissolve in water. A base is not defined by whether it dissolves, only by the fact that it reacts with an acid to give a salt and water only.