CNC Lathe Training Lesson 9: Measurement, Tolerances and Surface Quality

A CNC lathe program running flawlessly does not mean the produced part is necessarily correct. The program can say X40.0, yet the part measures Ø40.08. The cause can be tool wear, tool/part deflection, a wrong offset, a faulty measuring method, a hot part, chuck force, insert geometry, vibration, ball-screw/slide backlash or the machine’s loss of geometric accuracy. So the CNC operator must not only machine the part but verify its conformity to the drawing with a correct measuring method.

Measurement is not a simple check at the end of production; the result determines how the wear offset is changed, whether production continues, when the tool is changed, whether the error is in the program or the setup, and whether the part is accepted or rejected. In this lesson three topics are covered together: measurement (determining the real size), tolerance (whether the size is within acceptable limits) and surface quality (the functional suitability of the surface).

xF0x9Fx93xB7 IMAGE AREA — The trio of measurement, tolerance and surface quality
Suggested alt text: “CNC lathe measurement tolerance and surface quality”

1. Measurement, Checking and Measurement Uncertainty

Measurement is comparing a quantity with an accepted unit (e.g. “part diameter = 39.982 mm”). Checking determines whether the part meets the acceptance condition (e.g. “the GO gauge enters, the NO-GO gauge does not → acceptable”). A GO/NO-GO gauge does not give the exact diameter, only whether it is within the tolerance limits; so both numerical measuring and limit-checking tools are used. No measurement is fully error-free: resolution, calibration, applied force, surface cleanliness, temperature, surface roughness, measuring direction and tilting the instrument all affect the result. The same Ø40 shaft can be read by different people as 39.98–40.00; this difference does not always come from the part but from the method.

2. Resolution and Choosing the Instrument by Tolerance

Resolution is the smallest size change an instrument can show. Displaying 0.01 mm on screen does not mean it measures with 0.01 mm accuracy in every condition; real accuracy depends on the instrument quality, calibration and use.

As the tolerance narrows, a more precise instrument is needed. For Ø40 ±0.20 (total 0.40 mm) a caliper may be enough; for Ø40 ±0.01 (total 0.02 mm) a caliper is not reliable, a micrometer is needed. Practical rule: the instrument’s uncertainty/readability must be markedly smaller than the tolerance being checked (for a 0.02 mm tolerance, 0.01 mm resolution is borderline; there are only two display steps across the whole range).

Before measuring: stop the spindle completely, clean chips/burrs, wipe the surfaces, check the instrument zero, and assess whether the part is too hot. A rotating workpiece is not measured with a caliper/micrometer — the instrument can be thrown, the operator injured, and the part/instrument damaged.

xF0x9Fx93xB7 IMAGE AREA — Correct measuring technique with caliper and micrometer
Suggested alt text: “CNC lathe caliper and micrometer measuring”

3. Caliper and Micrometer

With a caliper you can measure OD/ID, length, step, groove width and hole depth. When measuring an OD, clean the jaws, check zero, place perpendicular to the part, close with light force and find the real diameter position with small movements. Tilting gives a larger/irregular value; over-clamping flexes a thin part — a caliper is not a clamping tool. In ID measurement the largest reading shows the true diameter direction; but a caliper is insufficient in deep/small holes and tight tolerances. On the depth rod, the body must seat fully on the reference surface, and the drill’s conical bottom must be separated from the cylindrical hole depth.

A micrometer is more precise than a caliper (the external micrometer is the most common). Measuring force is critical: the final contact is made with the ratchet/constant-force mechanism so every measurement is taken with the same force. Over-tightening the thimble crushes a thin part, flexes the frame and shows the size smaller than it is. Measuring a shaft at one point is not enough: along the length there can be taper, ovality or barrelling; a precision shaft is measured at the front, middle and near the chuck, and in the same section in two directions ~90° apart. A change along the length shows taper; a change with direction in the same section shows ovality.

4. ID Tools, Dial Indicator, Runout and Gauges

IDs are harder to measure; tools: internal micrometer, three-point internal micrometer (self-centres in the hole), telescopic gauge, bore dial gauge, air gauge and GO/NO-GO plug gauge. A bore dial gauge shows a difference relative to a set reference rather than an absolute value (e.g. zeroed with a Ø40.000 setting master and reading +0.012 on the part). A dial indicator measures small position differences; used for part/chuck runout, face wobble and second-setup centring. Runout = maximum − minimum (e.g. −0.01 to +0.03 → total 0.04 mm).

Runout is not the same as roundness: runout shows the combination of centre offset, roundness error, clamping error and shaft bend. 0.04 mm of runout does not mean the part is exactly 0.04 mm oval — the part can be round but clamped off-centre, or centred but oval. Gauges (plug, ring, snap, thread gauge, radius gauge, feeler) quickly check tolerance limits: the GO side must go, the NO-GO side must not; the gauge is not forced. A thread pitch gauge only checks the thread pitch (not the effective/root/major diameter or tolerance class); in series threading, GO/NO-GO ring (external) / plug (internal) gauges are used.

xF0x9Fx93xB7 IMAGE AREA — Tolerance, basic size/deviation and the fit system
Suggested alt text: “CNC lathe tolerance and fit system”

5. Tolerance, Deviation and Target Size

No part can be made exactly at the theoretical size; tolerance is the acceptable range of variation. For Ø40 ±0.05, the max is 40.05, the min 39.95; total tolerance 40.05 − 39.95 = 0.10 mm. Basic size, upper/lower deviation: Ø40 +0.02 / -0.01 → basic 40.00, upper +0.02, lower −0.01; limits 40.02 and 39.99; total tolerance 0.03 mm; acceptance range 39.990–40.020. One-sided tolerance: Ø30 +0.00 / -0.04 → 30.00–29.96 (no going above nominal). So saying “tolerance 0.04 mm” is not enough; which side of the basic size the tolerance zone is on also matters.

Target size: setting the size directly at a tolerance limit is risky. For Ø40 ±0.02 (39.98–40.02) the theoretical middle is 40.00. If the OD tends to grow as the tool wears, the target at the start is set near the tolerance centre or, in a controlled way, to the suitable side per process behaviour; starting the part very close to the upper limit like 40.019 is wrong (a small amount of wear pushes it out of tolerance).

6. ISO Tolerance, Fits and Geometric Tolerances

The ISO 286 system shows tolerance classes, deviations and fits for cylindrical elements (e.g. Ø40 H7, Ø40 h6, Ø40 H7/g6): the number is the basic size, the letter is the tolerance zone position relative to the zero line, the digit is the tolerance quality. An uppercase letter is a hole (H7), lowercase a shaft (h6); the exact deviation values come from the ISO table. A fit is the clearance/interference relationship of a shaft and hole after assembly:

If a dimension has no tolerance next to it, it is not unlimited; the general tolerance note in the drawing’s title block applies. A key distinction: dimensional tolerance only gives the size limit; even if the part measures Ø40.00 it can be oval, tapered, bent or off-centre. Geometric tolerances (roundness, cylindricity, parallelism, perpendicularity, concentricity, position, runout) control these. Roundness is a section’s conformity to the ideal circle (horizontal 40.00 / vertical 39.96 → 0.04 mm difference). Cylindricity/taper: the diameter should not change along the length (front 40.00 / middle 39.98 / rear 39.95 → shrinking near the chuck is taper; the cause can be tool/part deflection, tailstock setting, geometry, wear or mechanical backlash). Ball-screw/ball wear creates backlash and can spoil dimensional accuracy even when the CNC data looks normal; recurring errors not fixable by offset are sought not only in the program but in the mechanical system too.

Measurement temperature: metal expands when heated; the standard reference for industrial length measurement is 20 °C. A hot part just off the CNC can measure larger than at room temperature; large diameters, long cycles, dry machining, high speed and tight tolerances are sensitive to temperature. In precise production, the part is given time to stabilise, the instrument and part are kept in the same environment, and the measurement time is standardised.

xF0x9Fx93xB7 IMAGE AREA — Surface roughness (Ra), turning marks and measuring direction
Suggested alt text: “CNC lathe surface roughness Ra and turning marks”

7. Surface Roughness: Ra, Rz and Measurement

Even if a machined surface looks flat to the eye, at microscopic scale it has peaks, valleys, tool marks and waviness; these irregularities are called surface roughness, and they affect not only appearance but wear, friction, lubrication, sealing and heat/electrical conduction (critical for bearings, seal-running shafts, hydraulic pistons, sealing and fit surfaces). Ra is the arithmetic mean of the profile deviations over the evaluation length (unit µm; e.g. Ra 3.2 / 1.6 / 0.8). A smaller value is usually a smoother surface; but at the same Ra the peak shape, scratch direction and functional behaviour can differ. Rz is based on peak/valley heights and is more sensitive to pronounced differences; whichever parameter the drawing requires (e.g. Ra 1.6) is the one measured.

Measurement methods: comparison specimen (fast, approximate), stylus profilometer (the tip moves along the surface, Ra/Rz computed), optical systems. Measuring direction matters: on a turned surface the tool marks are mostly circumferential; the probe is moved in the direction that crosses the dominant marks (along the shaft axis) — the wrong direction shows a different roughness. On the drawing the surface symbol + value (e.g. Ra 3.2) and a general note (“unspecified surfaces Ra 3.2”) can appear.

8. Effect of Cutting Parameters on Surface Quality

Feed: as it increases, the spacing between tool marks grows and the surface roughens; experiments show Ra rises with feed (F0.30 → rough, F0.15 → good, F0.08 → fine mark). But an excessively low feed is not always good: if the tool cannot form enough chip, it rubs instead of cutting, causing sticking and fast wear. Nose radius: at the same feed a larger nose radius theoretically leaves smaller marks but increases cutting force and can vibrate a thin/long part. Theoretical roughness:

R_a ≈ f² ÷ (32 × r_ε)  |  e.g. f=0.20, r_ε=0.8 → 0.04 ÷ 25.6 = 0.00156 mm ≈ 1.56 µm

(f in mm/rev, r_ε in mm; result in mm, ×1000 for µm.) This is an ideal geometric estimate; the real value changes with vibration, material, wear, coating, sticking and rigidity. Speed/cutting speed: in a suitable range, higher speed can reduce built-up edge and improve the surface; but there is no “faster is always better” rule (excessive speed causes heat/wear/vibration). Depth of cut increasing raises force/deflection/vibration and roughness; but the finishing pass must not be too small either (leave an adequate, balanced allowance). Roughing uses high feed, finishing low feed.

Other factors: a worn insert spoils both size and roughness (dull surface, lines/burrs, vibration); insert change is planned not by full breakage but by signs like the size drifting one way, loss of surface gloss and rising load. Built-up edge (material sticking to the cutting edge) tears the surface and makes the size fluctuate; the fix is not only lowering the feed but evaluating cutting speed/insert grade/geometry/coolant together. Chatter (unstable vibration) leaves regular waves/cross marks; caused by part overhang, clamping, tool overhang, large radius, excessive pass and a dull tip. Suitable coolant reduces temperature and sticking and improves size stability, but if it reaches irregularly it can cause thermal shock.

xF0x9Fx93xB7 IMAGE AREA — Multi-point measurement along the shaft (taper/ovality) and first-part offset correction
Suggested alt text: “CNC lathe multi-point diameter measurement and offset correction”

9. First-Part Measurement Plan and Which Offset Fixes Which Error?

On the first part you do not take a few random measurements; the drawing features are measured in an order (total length, main ODs, IDs, step lengths, groove position/width, thread, chamfer/radius, runout, surface roughness) and each is written into a table:

Diameter correction: if the OD is large, the tool’s X wear offset is checked (target Ø40.00, measured Ø40.06 → in a diameter-based system ~-0.06; verify the diameter/radius interpretation). If the ID is small, the tool must go to a larger X. Length correction: if a shoulder/face is wrong, the tool’s Z wear offset; but if all tools’ lengths shift by the same amount, the workpiece Z zero/stop is checked. Program or offset? With an offset: tool wear, insert-change difference, a similar deviation on all diameters of the same tool. In the program: one shoulder/chamfer/radius/groove/thread length wrong. Setup/machine: random change each part, taper, high runout in the second setup, an error that returns despite the offset (ball-screw/slide backlash). Fixing one shoulder with the tool’s Z offset shifts all Z dimensions of that tool — then the program coordinate is fixed.

10. Measurement Frequency and the Size Trend in Series Production

Once the first part is correct, not measuring all parts is not always right; the frequency depends on tolerance width, wear rate, material variability, quantity and criticality. Approach: first part all critical features; 2nd–3rd critical diameters/lengths; if stable, at set intervals; re-do the first part after an insert change; full check at a setup change. Look not only at whether it is in tolerance but at which way the size is trending:

Instrument care: a measuring instrument is not used like a production tool (no scraping burrs with a caliper, no clamping a micrometer like a vice, not leaving it on a chippy machine, not dropping it); it is cleaned, stored in its box and calibrated regularly. A caliper must read 0.00 when closed; a micrometer is zeroed with a setting rod/gauge block — all measurements taken with a wrongly zeroed instrument are wrong in the same direction. A 0.02 mm chip/burr on the measuring surface completely changes the accept decision on a 0.02 mm-tolerance part; cleanliness is essential especially in IDs/grooves. A thin-walled part can look round while clamped and go oval when removed; it is measured with low force, in several directions and in the free state. In the second setup, the total length, concentricity between diameters and the reference-diameter runout are checked.

11. Analysing Dimensional Errors: Examples

• Diameter changing along the length (taper): Ø50 ±0.02 target, front 50.01 / middle 50.04 / near chuck 50.07 → entering -0.05 in the X offset is wrong; the offset shifts the whole surface equally, it does not remove the taper. First fix the cause (deflection/tailstock/geometry/cutting force).
• All diameters of the same tool large: T0101 → Ø40.06 / Ø50.05 / Ø60.06 (all ~the same amount) → X wear/geometry or insert-change difference; a small X wear correction is appropriate.
• Only one diameter wrong: Ø40→40.00 / Ø50→50.08 / Ø60→60.00 (all the same tool) → not the general X offset; check the Ø50 program coordinate/tool path/radius comp/vibration (changing the X offset spoils the correct diameters).
• Size right, surface rough: diameter in tolerance but Ra above the limit → lower the finishing feed, change insert/geometry, adjust speed, reduce vibration/overhang (changing the diameter by offset does not solve roughness).
• Bright surface but size varying: if 39.99 / 40.03 / 39.97 / 40.04 scatter, look for seating on the stop, chuck pressure, a loose holder, ball-screw/slide backlash, temperature or measurement inconsistency (gloss is not proof of process stability).

12. Practical Measuring Order and Safe Offset Correction

General order: stop the spindle, take the part to a safe position, clean chips/coolant, open the drawing/tolerances, check the instrument zero; measure the critical ODs first, then IDs, step/lengths, groove/thread, taper/ovality at different points, runout with a dial indicator if needed, and surface roughness; record the results, identify the error source (program/offset/tool/clamping/machine), make a small controlled correction and re-measure a new part.

Safe offset example: Ø40.000 ±0.015 (accept 39.985–40.015), first measurement 40.025 → 0.010 mm outside the upper limit. To approach the centre (40.000) instead of the lower limit, the diameter change needed is 40.025 − 40.000 = 0.025 mm; in a diameter-based X wear system the starting correction is ~-0.025. Safe approach: verify the offset is diameter-based, enter it on the correct tool row, check the INPUT/+INPUT difference, machine a new part, measure with the same instrument/method, record the final value. Habits that make results reliable: verify the same measurement at least twice, measure precise diameters at several points, keep the instrument perpendicular, clean, apply no excessive force, account for a hot part, precompute the limits, record the measurement, do not make big corrections at once, and investigate the cause instead of covering the same error with an offset.

Lesson Summary

• Even a correct program may not give a correct part; production is verified with measurement, tolerance and surface quality.
• The instrument is chosen by the tolerance; a rotating part is not measured, one point is not enough, and instrument zero and cleanliness are checked.
• Tolerance = acceptance range; basic size + upper/lower deviation give the limits. ISO H/h and fits (clearance/transition/interference) are evaluated per the table. A correct diameter can still have wrong geometry (oval/taper).
• Surface roughness is measured with Ra/Rz; feed, nose radius (Ra≈f²/32rε), speed, pass, wear, built-up edge and vibration affect it.
• An error is diagnosed first: offset (same tool, stable deviation), program (a single feature) or setup/machine (taper, random change). Offsets are corrected in small steps.

Assessment Test

1. What is the acceptance range for Ø40 +0.02 / -0.01? A) 39.98–40.02 B) 39.99–40.02 C) 39.99–40.01 D) 40.00–40.03
2. (True/False) Even if the diameter is in tolerance, the part can be oval or tapered; dimensional tolerance does not guarantee geometry.
3. What is the standard reference temperature for industrial length measurement? A) 0 B) 20 C) 25 D) 37 °C
4. With target OD Ø40.00 measured Ø40.06, roughly what is entered in the X wear offset in a diameter-based system? A) +0.06 B) -0.06 C) -0.03 D) 0
5. (True/False) If only one shoulder is out of position, fixing it with the tool’s Z offset is the best solution.
6. For feed f=0.2 mm/rev and nose radius rε=0.8 mm, what is the theoretical Ra in µm? A) 0.16 B) 1.56 C) 15.6 D) 156
7. What does a thread pitch gauge measure? A) Effective diameter B) The thread pitch C) Tolerance class D) Root diameter

Open-Ended Questions

• Explain the difference between measurement and checking.
• Why is measuring a shaft at one point not enough?
• Explain clearance, transition and interference fits with examples.
• Explain the effect of feed and nose radius on surface roughness.
• How do you decide whether a dimensional error is fixed by offset, program or setup?

Completing the Basic CNC Lathe Training

With this lesson, the nine-part Mentor CNC Basic CNC Lathe Training is complete. The student no longer only reads the program; they can evaluate the produced part’s diameter, length, internal dimensions, runout, tolerance limits, fit condition and surface quality with a correct method, and from the measurement result tell whether the error comes from the program, offset, tool, clamping or machine.

The chain built across the series is complete: machine anatomy and safety → axes and coordinates → cutting tools → cutting values → workholding and offsets → basic programming → canned cycles → program management and data transfer → measurement, tolerances and surface quality. But completing the theory does not mean independent production can be done straight away on a real machine; the control panel, referencing, tool measurement, first-part setup and measurement practice must be reinforced under the supervision of an authorised trainer or experienced operator. With Mentor CNC, your journey of carrying theory to the shop floor reaches a strong foundation here.