MEASURED
QUANTITY, INSTRUMENT
OR GAUGE |
RANGE (AND
SPECIFICATION WHERE APPROPRIATE) |
BEST MEASUREMENT
CAPABILITY EXPRESSED AS
AN UNCERTAINTY (±) |
REMARKS |
Bandwidth
Amplitud Relative
to 50 kHz |
50
kHz to 100 MHz |
2.2%
of reading |
Fluke
5520A/SC600 |
100
MHz to 300 MHz |
2.7%
of reading |
300
MHz to 600 MHz |
4.9%
of reading |
Electrical
Calibration
of RTD’s
Pt 385,
100 Ω |
-
200 °C to – 80 °C |
0.05
°C |
Fluke
5520A/SC600 |
-
80 °C to 0 °C |
0.05
°C |
0
°C to 100 °C |
0.07
°C |
100
°C to 300 °C |
0.09
°C |
300
°C to 400 °C |
0.10
°C |
400
°C to 630 °C |
0.12
°C |
630
°C to 800 °C |
0.23
°C |
Electrical
Calibration
of RTD’s
Pt 3926,
100 Ω |
-
200 °C to – 80 °C |
0.05
°C |
Fluke
5520A/SC600 |
-
80 °C to 0 °C |
0.05
°C |
0
°C to 100 °C |
0.07
°C |
100
°C to 300 °C |
0.09
°C |
300
°C to 400 °C |
0.10
°C |
400
°C to 630 °C |
0.12
°C |
Electrical
Calibration
of RTD’s
Pt 3916,
100 Ω |
-
200 °C to - 190 °C |
0.25
°C |
Fluke
5520A/SC600 |
-
190 °C to – 80 °C |
0.04
°C |
-
80 °C to 0 °C |
0.05
°C |
0
°C to 100 °C |
0.06
°C |
100
°C to 260 °C |
0.07
°C |
260
°C to 300 °C |
0.08
°C |
300
°C to 400 °C |
0.09
°C |
400
°C to 600 °C |
0.10
°C |
600
°C to 630 °C |
0.23
°C |
Electrical
Calibration
of RTD’s
Pt 385,
200 Ω |
-
200 °C to - 190 °C |
0.04
°C |
Fluke
5520A/SC600 |
-
80 °C to 0 °C |
0.04
°C |
0
°C to 100 °C |
0.04
°C |
100
°C to 260 °C |
0.05
°C |
260
°C to 300 °C |
0.12
°C |
300
°C to 400 °C |
0.13
°C |
400
°C to 600 °C |
0.14
°C |
600
°C to 630 °C |
0.16
°C |
|
MEASURED
QUANTITY, INSTRUMENT
OR GAUGE |
RANGE (AND
SPECIFICATION WHERE APPROPRIATE) |
BEST MEASUREMENT
CAPABILITY EXPRESSED AS
AN UNCERTAINTY (±) |
REMARKS |
DC
Voltage-Measure |
10
kV to 20 kV |
2.3
% of reading |
Fluke
80K-40 with HP 3458A |
20
kV to 35 kV |
1.5
% of reading |
35
kV to 40 kV |
2.3
% of reading |
DC
Current – Generate |
10
µA to 1 A |
0.02
% |
Fluke
5520A/6
w/
34330 A |
1
A to 20.5 A |
0.18
% |
DC
Current – Measure |
100 pA to 100 µA |
24
µA/A + 0.8 nA |
HP
3458A with
Weston
shunt |
0.1 mA to 1 mA |
24
µA/A + 5 nA |
1 mA to 10 mA |
24
µA/A 50 nA |
10 mA to 100 mA |
41
µA/A + 0.5 µA |
0.1
A to 1 A |
0.013
% + 10 µA |
1
A to 600 A |
0.29
% of reading |
DC
Resistance – Generate |
0
Ω to 10 Ω |
450
µΩ/Ω + 0.001 Ω |
Fluke
5520A/6 |
10
Ω to 100 Ω |
3.3 mΩ/Ω + 0.001 4 Ω |
100
Ω to 1 kΩ |
30 mΩ/Ω + 0.002 Ω |
1 kΩ to 10 kΩ |
0.30
Ω/Ω + 0.02 Ω |
10 kΩ to 100 kΩ |
3
Ω/Ω + 0.2 Ω |
100 kΩ to 1 000 kΩ |
0.037
% + 2 Ω |
1
MΩ to 10 MΩ |
0.015
% + 50 Ω |
10
MΩ to 100 MΩ |
0.072
% + 3 000 Ω |
DC
Resistance- Measure |
0.1
Ω to 10 Ω |
19
µΩ/Ω + 0.05 mΩ |
HP
3458A |
10
Ω to 100 Ω |
15
µΩ/Ω+ 0.5 mΩ |
0.1 kΩ to 1 kΩ |
13
µΩ/Ω+ 0.5 mΩ |
1 kΩ to 10 kΩ |
13
µΩ/Ω+ 5 mΩ |
10 kΩ to 100 kΩ |
13
µΩ/Ω+ 0.05 Ω |
0.1
MΩ to 1 MΩ |
24
µΩ/Ω+ 2 Ω |
1
MΩ to 10 MΩ |
65
µΩ/Ω+ 100 Ω |
10
MΩ to 100 MΩ |
0.058
% + 1 kΩ |
0.1
GΩ to 1.2 GΩ |
0.58
% + 10 k Ω |
|
MEASURED
QUANTITY, INSTRUMENT
OR GAUGE |
RANGE (AND
SPECIFICATION WHERE APPROPRIATE) |
BEST MEASUREMENT
CAPABILITY EXPRESSED AS
AN UNCERTAINTY (±) |
REMARKS |
AC
Voltage – Generate
45
Hz to 10 kHz |
1
mV to 330 mV |
0.026
% |
Fluke
5520A/6 |
330
mV to 3.3 V |
0.054
% |
3.3
V to 33 V |
0.026
% |
33
V to 330 V |
0.042
% |
330
V to 1 000 V |
0.033
% |
AC
Voltage- Measure
3
µV to 10 mV |
1
Hz to 40 Hz |
0.03
% + 3 µV |
HP
3458 A
Synchronous
sub-sampled mode |
40
Hz to 1 kHz |
0.02
% + 1.1 µV |
1
kHz to 20 kHz |
0.03
% + 1.1 µV |
20
kHz to 50 kHz |
0.12
% + 1.1 µV |
50
kHz to 100 kHz |
0.58
% + 1.1 µV |
100
kHz to 300 |
4.6
% + 2 µV |
AC
Voltage- Measure
10
mV to 100 mV |
1
Hz to 40 Hz |
80
µV/V + 4 µV |
HP
3458 A
Synchronous
sub-sampled mode |
40
Hz to 1 kHz |
80
µV/V + 2 µV |
1
kHz to 20 kHz |
0.02
% + 2 µV |
20
kHz to 50 kHz |
0.03
% + 2 µV |
50
kHz to 100 kHz |
0.09
% + 2 µV |
100
kHz to 300 |
0.35
% + 10 µV |
0.3
MHz to 1 MHz |
1.2
% + 10 µV |
1
MHz to 2 MHz |
1.7
% + 10 µV |
AC
Voltage- Measure
0.1
V to 1 V |
1
Hz to 40 Hz |
80
µV/V + 40 µV |
HP
3458 A
Synchronous
sub-sampled mode |
40
Hz to 1 kHz |
80
µV/V + 20 µV |
1
kHz to 20 kHz |
0.02
% + 20 µV |
20
kHz to 50 kHz |
0.03
% + 20 µV |
50
kHz to 100 kHz |
0.09
% + 20 µV |
100
kHz to 300 |
0.35
% + 100 µV |
0.3
MHz to 1 MHz |
1.2
% + 100 µV |
1
MHz to 2 MHz |
1.7
% + 100 µV |
|
MEASURED
QUANTITY, INSTRUMENT
OR GAUGE |
RANGE (AND
SPECIFICATION WHERE APPROPRIATE) |
BEST MEASUREMENT
CAPABILITY EXPRESSED AS
AN UNCERTAINTY (±) |
REMARKS |
AC
Voltage- Measure
1
V to 10 V |
1
Hz to 40 Hz |
80
µV/V + 0.4 mV |
HP
3458 A
Synchronous
sub-sampled mode |
40
Hz to 1 kHz |
80
µV/V + 0.2 mV |
1
kHz to 20 kHz |
0.02
% + 0.2 mV |
20
kHz to 50 kHz |
0.03
% + 0.2 mV |
50
kHz to 100 kHz |
0.09
% + 0.2 mV |
100
kHz to 300 |
0.35
% + 1 mV |
0.3
MHz to 1 MHz |
1.2
% + 1 mV |
1
MHz to 2 MHz |
1.7
% + 1 mV |
AC
Voltage- Measure
10
V to 100 V |
1
Hz to 40 Hz |
0.02
% + 4 mV |
HP
3458 A
Synchronous
sub-sampled mode |
40
Hz to 1 kHz |
0.02
% + 4 mV |
20
kHz to 50 kHz |
0.04
% + 2 mV |
50
kHz to 100 kHz |
0.14
% + 2 mV |
100
kHz to 300 |
0.46
% + 10 mV |
0.3
MHz to 1 MHz |
1.7
% + 10 mV |
AC
Voltage- Measure
100
V to 700 V |
1
Hz to 40 Hz |
0.05
% + 40 mV |
HP
3458 A
Synchronous
sub-sampled mode |
40
Hz to 1 kHz |
0.05
% + 20 mV |
1
kHz to 20 kHz |
0.07
% + 20 mV |
20
kHz to 50 kHz |
0.14
% + 20 mV |
RF
Power Generate
DC
to 20 MHz |
23.98 dBm to –56.02 dBm |
0.5
dB |
HP
3458 A
Synchronous
sub-sampled mode |
RF
Power Generate
20
MHz to 2 GHz |
10 dBm to – 127 dBm |
1.3
dB |
HP
3458 A
Synchronous
sub-sampled mode |
RF
Power Generate
2
GHz to 6.6 GHz |
10 dBm to –30 dBm |
0.2
dB |
HP
3458 A
Synchronous
sub-sampled mode |
-30 dBm to –60 dBm |
0.5
dB |
HP
3325A |
-60 dBm to –70 dBm |
0.96
dB |
-70 dBm to –80 dBm |
3.1
dB |
HP3335A |
-80 dBm to –90 dBm |
3.2
dB |
-90 dBm to –100 dBm |
3.3
dB |
Agilent
E8275D |
|
MEASURED
QUANTITY, INSTRUMENT
OR GAUGE |
RANGE (AND
SPECIFICATION WHERE APPROPRIATE) |
BEST MEASUREMENT
CAPABILITY EXPRESSED AS
AN UNCERTAINTY (±) |
REMARKS |
RF
Power-Generate
6.6
GHz to 12.3 GHz |
10 dBm to –30 dBm |
0.20
dB |
Agilent
E8275D |
-30 dBm to 60 dBm |
0.50
dB |
-60 dBm to –70 dBm |
0.96
dB |
-70 dBm to –80 dBm |
3.4
dB |
-80 dBm to –90 dBm |
3.5
dB |
-90 dBm to –100 dBm |
3.6
dB |
RF
Power-
Generate
12.3
GHz to 18.6 GHz |
10 dBm to –30 dBm |
0.20
dB |
Agilent
E8275D |
-30 dBm to 60 dBm |
0.50
dB |
-60 dBm to –70 dBm |
0.96
dB |
-70 dBm to –80 dBm |
4.2
dB |
-80 dBm to –90 dBm |
4.4
dB |
-90 dBm to –100 dBm |
4.7
dB |
RF
Power-
Generate
18.6
GHz to 26 GHz |
10 dBm to –10 dBm |
0.30
dB |
Agilent
E8275D |
-10 dBm to –20 dBm |
0.36
dB |
-20 dBm to –30 dBm |
0.38
dB |
-30 dBm to –40 dBm |
0.50
dB |
-40 dBm to –50 dBm |
0.52
dB |
-50 dBm to –60 dBm |
0.58
dB |
-60 dBm to –70 dBm |
0.96
dB |
-70 dBm to –80 dBm |
5.1
dB |
-80 dBm to –90 dBm |
5.4
dB |
-90 dBm to –100 dBm |
5.6
dB |
RF
Power- Measure
100
kHz to 26.5 GHz
2.6
GHz to 26.5 GHz |
30 dBm to –20 dBm |
2.9
% |
Agilent
E8275D |
20 dBm to –70 dBm |
2.9
% |
RF
Attenuation-Tuned RF Power Measure
100
kHz to 26.5 GHz |
0 dBm to –10 dBm |
0.03
dB |
HP
8902A
HP
11722A
HP
11792A
HP
11793 A |
-10 dBm to –20 dBm |
0.06
dB |
-20 dBm to –30 dBm |
0.08
dB |
-30 dBm to –40 dBm |
0.11
dB |
-40 dBm to –50 dBm |
0.14
dB |
-50 dBm to –60 dBm |
0.16
dB |
-60 dBm to –70 dBm |
0.20
dB |
-70 dBm to –80 dBm |
0.22
dB |
|
RF
Attenuation-Tuned RF Power Measure
100
kHz to 26.5 GHz |
-80 dBm to –90 dBm |
0.22
dB |
HP
8902A
HP
11722A
HP
11792A
HP
11793 A |
-90 dBm to –100 dBm |
0.24
dB |
-100 dBm to –110 dBm |
0.32
dB |
-110 dBm to –120 dBm |
0.37
dB |
|
|
Dimensional Capabilities
|
MEASURED
QUANTITY, INSTRUMENT
OR GAUGE |
RANGE (AND
SPECIFICATION WHERE APPROPRIATE) |
BEST MEASUREMENT
CAPABILITY EXPRESSED AS
AN UNCERTAINTY (±) |
REMARKS |
Micrometer,
Height, and Depth Gages |
0
in to 24 in |
(67
+ 1.6L) µin |
Gage
Blocks |
Calipers |
0
in to 24 in |
(119
+ 15L) µin |
Gage
Blocks |
Angle
Blocks |
0.25°
to 45° |
0.001
0° |
Gage
Blocks/Sine Bar, Electronic Probe |
45°
to 90° |
0.001
0° |
Dial
Indicators |
0
in to 2 in |
(70
+ 1.6L) µin |
Gage
Blocks |
Pin
Gages |
0
in to 4 in |
(19
+ 15L) µin |
Gage
Blocks |
Rulers/Tape
Measure |
Up
to 25 ft |
0.009
0 in |
Gage
Blocks with Calipers |
Thread
Gages (4 TPI to 96 TPI) |
0
in to 2 in |
88 µin |
Helios
& Thread Wires |
Levels |
Up
to 48 in |
(0.6R
+ 500L) µin |
Master
precision level and surface plate |
Wire
Cloth and Sieves |
0.15
mm to 12.5 mm |
5.2
µm |
Optical
Comparator |
Optical
Comparators-Linearity (bed travel indication) |
0
in to 12 in |
2
000 µin |
Glass
Scale, Spherical Ball Set,
Gage
Blocks |
Conductivity-Measuring
Equipment |
45
µS, 450 µS, 1 500 µS, 4 500 µS |
1.2
% of reading |
Standard
Solutions |
pH-Measuring
Equipment |
4
pH, 7 pH, 10 pH |
0.020
pH units |
Standard
Solutions |
Viscosity
Meter Calibration |
To
33 % of Scale |
2.7
% |
Viscosity
Solutions |
33
% to 66 % of Scale |
1.8
% |
66
% to 100 % of Scale |
1.7
% |
|
Scales
and Balances |
1
mg to 100 mg |
0.026
mg |
Class
1 Weights |
10
mg to 500 mg |
0.043
mg |
1
g to 50 g |
0.15
mg |
50
g to 300 g |
0.90
mg |
300
g to 500 g |
1.7
mg |
0.5
kg to 1 kg |
3.7
mg |
1
kg to 3 kg |
8
mg |
3
kg to 5 kg |
12
mg |
5
kg to 10 kg |
34
mg |
1
lb to 5 lb |
0.000
60 lb |
Class
F Weights |
3
lb to 10 lb |
0.001
lb |
10
lb to 25 lb |
0.004
lb |
25
lb to 50 lb |
0.006
lb |
50
lb to 100 lb |
0.02
lb |
100
lb to 600 lb |
0.02
lb |
Force-Load
Cells/Force Gauges Tension |
0 lbf to 600 lbf |
0.98 lbf |
Class
F Weights |
|
|
Mechanical Capabilities
|
MEASURED
QUANTITY, INSTRUMENT
OR GAUGE |
RANGE (AND
SPECIFICATION WHERE APPROPRIATE) |
BEST MEASUREMENT
CAPABILITY EXPRESSED AS
AN UNCERTAINTY (±) |
REMARKS |
Torque Tranducers |
0
ft-lb to 1 000 ft-lb |
0.26
% of reading |
AKO
TSD –0251 torque system |
Torque
Wrenches |
0
ft-lb to 1 000 ft-lb |
0.19
% of reading |
AKO
TSD –0251 torque system |
RPM
Measuring Equipment |
0
rpm to 100 000 rpm |
0.59
% of reading |
Agilent
53132A |
Volumetric
Flow, Liquids |
0
lb/min to 1 000 lb/min |
0.03
% of rate + 0.1 lb/min |
D100
Flow sensor |
Gas
Flow |
5 sccm to 500 sccm |
0.29
% of Full Scale |
SL-800
Gas Flow Calibrator |
50 sccm to 5 000 sccm |
0.29
% of Full Scale |
Gas
Concentration Alarms-Carbon Monoxide (CO) |
25 pppm |
2.8 ppm |
Standard
Gases |
50 ppm |
5.5 ppm |
Gas
Concentration Alarms-Sulfur Dioxide (SO2) |
10 ppm |
2 ppm |
|
Air
Velocity-Measure |
25
ft/min to 300 ft/min |
1.8
% of reading 1 fpm |
TSI
8386 |
300
ft/min to 500 ft/min |
1.5
% of reading |
500
ft/min to 2 000 ft/min |
1.6
% of reading |
2
000 ft/min to 2 800 ft/min |
1.8
% of reading |
2
800 ft/min to 4 500 ft/min |
2.2
% of reading |
4
500 ft/min to 7 000 ft/min |
1.7
% of reading |
7
000 ft/min to 9 500 ft/min |
1.2
% of reading |
|
|
Thermodynamic Capabilities
|
MEASURED
QUANTITY, INSTRUMENT
OR GAUGE |
RANGE (AND
SPECIFICATION WHERE APPROPRIATE) |
BEST MEASUREMENT
CAPABILITY EXPRESSED AS
AN UNCERTAINTY (±) |
REMARKS |
Temperature-Measure |
-200
°C to 100 °C |
0.036
°C |
Hart
1502A with 5699 RTD |
100
°C to 660 °C |
0.061
°C |
Relative
Humidity- Measuring |
5
% RH to 95 % RH |
0.62
% RH |
Vaisala HMI-41 and chamber |
Liquid-In-Glass
Thermometers |
-70
°C to 110 °C |
0.03
°C + 0.25 R |
Liquid
Bath and PRT |
28
°C to 300 °C |
0.04
°C + 0.25 R |
IR
Temperature-
Measuring
Equipment Blackbody Sources |
Ambient
to 285 °C |
0.46
°C + 0.016 °C/°C |
Blackbody
Source, IR Thermometer |
285
°C to 870 °C |
3.1
°C + 0.005 °C/°C |
|
|
Time and Frequency Capabilities
|
MEASURED
QUANTITY, INSTRUMENT
OR GAUGE |
RANGE (AND
SPECIFICATION WHERE APPROPRIATE) |
BEST MEASUREMENT
CAPABILITY EXPRESSED AS
AN UNCERTAINTY (±) |
REMARKS |
Frequency-
Measuring Equipment |
0.01
Hz to 225 MHz |
2.5
µHz/Hz + 5µHz |
Agilent
53132A |
Timers/Stopwatches |
5
s to 24 hr |
0.04
s |
Agilent
53132A |
Phase
Modulation- Measure
200
Hz to 10 kHz |
150
kHz to 10 MHz |
4
% of reading + 1 digit |
HP
8902A w/ 11793A |
10
MHz to 1.3 GHz |
3%
of reading + 1 digit |
Phase
Modulation- Measure
200
Hz to 20 kHz |
1.3
GHz to 26.5 GHz |
4
% of reading + 1 digit |
HP
8902A w/ 11793A |
Amplitude
Modulation-Measure
Rate:
50 Hz to 10 kHz
Depths:
5 % to 99 % |
150
kHz to 10 MHz |
2
% of reading + 1 digit |
HP
8902A w/ 11793A |
Amplitude
Modulation-Measure
Rate:
20 Hz to 10 kHz
Depths:
to 99 % |
150
kHz to 10 MHz |
3
% of reading + 1 digit |
HP
8902A w/ 11793A |
Amplitude
Modulation-Measure
Rate:
50 Hz to 50 kHz
Depths:
5 % to 99 % |
10
MHz to 1.3 GHz |
1
% of reading + 1 digit |
HP
8902A w/ 11793A |
Amplitude
Modulation-Measure
Rate:
20 Hz to 100 kHz
Depths:
to 99 % |
10
MHz to 1.3 GHz |
3
% of reading + 1 digit |
HP
8902A w/ 11793A |
Amplitude
Modulation-Measure
Rate:
20 Hz to 100 kHz
Depths:
5 % to 99 % |
1.3
GHz to 26.5 GHz |
1.5
% of reading + 1 digit |
HP
8902A w/ 11793A |
Amplitude
Modulation-Measure
Rate:
20 Hz to 100 kHz
Depths:
to 99 % |
1.3
GHz to 26.5 GHz |
3
% of reading + 1 digit |
HP
8902A w/ 11793A |
Amplitude
Modulation-Generate
Rate:
50 Hz to 100 kHz
Depths: 5 % to 99 % |
10
kHz to 0.99 GHz |
0.1
% |
HP
8656B |
Frequency
Modulation- Measure
Rate
20 Hz to 10 kHz
Dev.: <40 kHz peak |
250
kHz to 10 MHz |
2
% of reading + 1 digit |
HP
8902A w/ 11793A |
Frequency
Modulation- Measure
Rate
50 Hz to 100 kHz
Dev.: <400 kHz peak |
10
MHz to 1.3 GHz |
1
% of reading + 1 digit |
HP
8902A w/ 11793A |
Frequency
Modulation- Measure
Rate
20 Hz to 200 kHz
Dev.: <400 kHz peak |
10
MHz to 1.3 GHz |
5
% of reading + 1 digit |
HP
8902A w/ 11793A |
Frequency
Modulation- Measure
Rate
50 Hz to 100 kHz
Dev.: <400 kHz peak |
1.3
GHz to 26.5 GHz |
1
% of reading + 1 digit |
HP
8902A w/ 11793A |
Frequency
Modulation- Measure
Rate
20 Hz to 200 kHz
Dev.: <400 kHz peak |
1.3
GHz to 26.5 GHz |
5
% of reading + 1 digit |
HP
8902A w/ 11793A |
|