Metrology: The Science of Measurement

Introduction

Being able to take accurate measurements is essential to good machine tool work. In fact, without this ability, there really isn't an ability to make machines. One can almost chart the progress of machining as the progress of developing increasingly accurate measuring tools. This page summarizes the relative accuracies and types of tools that one might expect to find in machine shops. Tools are grouped by the relative magnitude of their precision and then by the type of measurement being made:

Marking Out: Tools designed for marking work at a particular level of precision. Marking out is only worth doing for 0.001" or less accurate work as it becomes impossible to position relative to the marks at much finer granularity than this.

Lengths/Heights/Depths & X, Y Locations: The most common measurements involve length in some linear direction. Lengths can be used either to determine how much to cut, or how to find a location relative to some other feature (X, Y Locations).

Outside Diameter: Measuring the outer diameter of a cylindrical piece is extremely common. Multiple measurements at different places and orientations can also determine taper, roundness, and other characteristics of cylinders.

Inside Diameter: Measuring the inside diameter of a hole is extremely common. Multiple measurements at different places and orientations can also determine taper, roundness, and other characteristics of the hole. For a variety of reasons, not the least of which is accessibility, inner diameters are harder to take than most other measurements.

Angles & Squareness: Squareness is the most common measurement of angles, but the need to align pieces at arbitrary angles often presents itself as well.

Some readers may feel they can achieve greater precision than I've indicated with some of these tools. That may well be true, but I'm trying to show where the "sweet spot" of performance for the tool is. Another important rule of thumb is that your measuring tools need to be at least 2x and preferably 5x to 10x finer in resolution than the precision you need to achieve in your parts.

Coarse Measurement: Greater than 0.001" error

Errors greater than a thousandth start to become coarse measurements for machine work. Inevitably it costs money for each greater level of precision achieved, so there will always be cases where coarse measurements suffice. Hobby class work begins at 0.020", and if you can consistently hit 0.010" you can build most hobby projects except for engines and precision tooling. Once you hit 0.005", some real machine work is possible and your parts will begin to have a finished appearance.

I find tools in this category to be occasionally useful, but usually for things like figuring out what size to cut rough stock on the saw, or perhaps where to casually locate an oversized hole.

0.001: Thousandths

Accuracy to a thousandth is the right neighborhood for the majority of machine work. You can build about any model engine or complex tool with the exception of turbines and other very high rpm close tolerance work.  For me, this is an ideal target and I find I can hold work to 0.001” in my manual machine work if I’m careful and think about what I’m doing.  You will probably have to do some careful adjustment on your Asian tools as well as have the right measuring instruments and good techniques to achieve this goal.

0.0001: Tenths or Better

This is the realm of real precision for machinists. If you can accomplish everything with tenths accuracy, you will seldom be called on to do better unless you're making precision optics or something similar. Tenths accuracies are usually achieved through grinding machines rather than by cutting. If your machines can do this reliably, they are well set up and in good condition and you should be doing something other than trying to learn anything from me. In fact, please drop me a note if you have any thoughts about what I can learn from you!