Scale is not dimension. A scale of 1 on all axes means this is the "natural" dimension of the underlying mesh from which the object is built. If you bring up the sidepanel in the 3D Viewport by pressing N, in the Item tab you will find the Scale and below that the Dimensions.

The default cube has an edge length of 2 m, like almost all mesh primitives in Blender have a size of 2 m in at least one axis. This is because they base the size on a unit circle which in mathematics has a radius of 1 and thus a diameter of 2.
But this is rather a dimensionless size of 2 "Blender units". The Unit System settings in the Scene Properties which you already discovered are what translates them into real world dimensions. You could set it to None, then the cube would just be 2 Blender units in size. By default it's Metric, which is why you get 2 m.
However, if you want to work on small models with real life dimensions but don't want to have all these tiny decimal numbers, you can switch the Unit Scale for example to 0.001 which you did (and Length from Meters to Milimeters). So without changing the object or its mesh, the cube now is only 2 mm instead of 2 m.

But that's all got nothing to do with the scale of an object. The scale is a multiplying factor to change the apparent dimensions of an object relative to the original size of the underlying mesh.
So by default all objects that you add to a scene have a Scale of 1 (no unit) on each axis, because they are naturally added in their original, unaltered size. In Blender these are usually Dimensions of 2 m because of the unit circle etc.
To change the Dimensions you can either scale an object in Object Mode, let's say by a factor of 2 which you can achieve by pressing S, 2, Return for example. Or you change the scale by entering new values for the Dimensions like 4 m for example, and the Scale will automatically be changed to 2 as well.

Now the cube has a Scale of 2 and the Dimensions are shown as 4 m for each axes. However, this is just a multiple of the original size. If you would go into Edit Mode and select an edge with the Edge Length overlays enabled, it would show a length of 2 m, although the dimensions in Object Mode show 4 m. That's because the original mesh is not changed, it is only shown translated to a multiple of its size.

To change the original mesh size from 2 m to 4 m you can now apply the scale in Object Mode hy pressing Ctrl+A > Apply > Scale. This way the scale will now be shown as 1, but the cube stays at 4 m length and the edge in Edit Mode will now show 4 m as well.


You could have done the scaling in Edit Mode directly. Scaling in Edit Mode instead of Object Mode changes the original mesh size while keeping the "outside" scale at 1. This scaling in Edit Mode as well as the other method, using Object Mode and then applying the scale, makes a permanent change in the mesh. The new size is now the basic original size and you cannot simply reverse it by setting the scale back to 1.
However, applying the scale or directly changing the mesh is important for many modifiers you can use on an object. Tools like the Bevel modifier use the original mesh size as relation when you for example set a bevel offset of 0.1 m - this will be 0.1 m on the original mesh with a length of 2 m, not the scaled object with a length of 4 m. In this case the bevel might look incorrect as it seems to be 0.2 m instead of the chosen value. It becomes even worse if the object is scaled unevenly on the three axes, which then creates an uneven bevel offset as well. So applying a scale when using object modifiers is very important, but also working on a mesh in Edit Mode can get weird results if the scale of an object is not 1.

