The pH scale is not a simple straight-line scale. Each step of one pH unit is a change of ten times in the concentration of hydrogen ions. When the hydrogen ion concentration increases by a factor of ten, the pH value decreases by one.
So a solution of pH 3 has ten times the hydrogen ion concentration of a solution of pH 4. A solution of pH 2 has 100 times the concentration of a solution of pH 4, because it is two units lower and 10 × 10 = 100. A solution of pH 1 has 1000 times the hydrogen ion concentration of a solution of pH 4.
It works the other way too. If a solution is diluted so that its hydrogen ion concentration falls by a factor of ten, its pH goes up by one, so the pH is higher. Diluting a pH 2 acid ten times gives a pH of 3. Diluting an acid never quite makes it neutral, because the pH moves towards 7 but stays on the acidic side.