In a non-enclosed system the reaction vessel is open to the air, so gases can enter or leave. The law of conservation of mass still holds, but the mass you measure on a balance can seem to change because you are not weighing every substance.
If a gas is made and escapes, the measured mass goes down. When calcium carbonate is heated in an open tube, carbon dioxide gas escapes, so the solid left behind weighs less. The same happens when acid is added to marble chips in an open flask. If a gas from the air joins a solid, the measured mass goes up. When magnesium burns in an open crucible, oxygen from the air joins the magnesium to make magnesium oxide. Magnesium of mass 0.24 g might become 0.40 g of magnesium oxide, so 0.16 g of oxygen has joined it.
The particle model explains this. Gas particles are far apart and move freely, so they can leave an open container or come in from the air. The atoms are not lost or created. If the reaction is carried out in a sealed container, no particles can escape or enter, and the total mass stays the same.
