[flow-tools] flow-capture reports PDUs out-of-sequence w/Juniper

Tue, 11 Sep 2001 10:47:47 -0500

--ZPt4rx8FFjLCG7dd
Content-Type: text/plain; charset=us-ascii

We're currently running JunOS 5.0R1.4 on a Juniper M10 exporting
version 5 "cflowd" PDUs to flow-capture 0.53, and I'm graphing the
results here:

   http://wwwstats.net.wisc.edu/

The problem I've been seeing is that flow-capture keeps reporting our
of sequence cflow/NetFlow PDUs:

   ftpdu_seq_check(): fail, expecting=113448924 received=113454940 lost=6016

After closer examination of the received sequence numbers the logs (see
attached "flow-capture.syslog"), I noticed that the problem is not that
the PDUs are being lost, but that they appear to be arriving
out-of-order.

I didn't notice this problem when collecting v5 NetFlow from a Cisco
75xx... and don't know of a reason why the UDP packets would be
reordered in transit.

Anyone know if this might be a problem at the source - with the
Juniper?  The only interesting experiment I can think of is directly
attach the collector box (running flow-capture) to an ethernet on the
Juniper, to eliminate the possibility of the PDUs being reordered in
transit.

Thanks,
Dave

-- 
plonka@doit.wisc.edu  http://net.doit.wisc.edu/~plonka  ARS:N9HZF  Madison, WI

--ZPt4rx8FFjLCG7dd
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="flow-capture.syslog"

Sep 10 06:41:11 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93259393 received=93261539 lost=2146
Sep 10 07:01:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93392971 received=93397541 lost=4570
Sep 10 07:05:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93432030 received=93432059 lost=29
Sep 10 07:05:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93432117 received=93432146 lost=29
Sep 10 07:05:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93432233 received=93432262 lost=29
Sep 10 07:05:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93432349 received=93432378 lost=29
Sep 10 07:05:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93432436 received=93432465 lost=29
Sep 10 07:05:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93432494 received=93432523 lost=29
Sep 10 07:05:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93432581 received=93432668 lost=87
Sep 10 07:05:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93432726 received=93432755 lost=29
Sep 10 07:05:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93432784 received=93432813 lost=29
Sep 10 07:05:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93432871 received=93432900 lost=29
Sep 10 07:05:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93432958 received=93432987 lost=29
Sep 10 07:05:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93433016 received=93433074 lost=58
Sep 10 07:15:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93504306 received=93508221 lost=3915
Sep 10 07:15:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93508308 received=93508337 lost=29
Sep 10 07:15:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93508395 received=93508424 lost=29
Sep 10 07:25:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93586567 received=93586596 lost=29
Sep 10 07:25:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93586712 received=93586567 lost=4294967150
Sep 10 07:25:12 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93586596 received=93586712 lost=116
Sep 10 08:00:13 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=93895283 received=93900793 lost=5510
Sep 10 08:25:14 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=94162879 received=94163169 lost=290
Sep 10 08:35:13 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=94274962 received=94276441 lost=1479
Sep 10 09:00:14 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=94557563 received=94566466 lost=8903
Sep 10 09:11:14 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=94692298 received=94694726 lost=2428
Sep 10 09:15:14 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=94749776 received=94752792 lost=3016
Sep 10 09:20:14 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=94824864 received=94824951 lost=87
Sep 10 09:21:14 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=94833685 received=94833714 lost=29
Sep 10 09:21:14 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=94833772 received=94833685 lost=4294967208
Sep 10 09:21:14 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=94833714 received=94833772 lost=58
Sep 10 09:25:14 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=94883824 received=94891132 lost=7308
Sep 10 09:46:14 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=95153610 received=95160317 lost=6707
Sep 10 09:55:14 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=95286372 received=95289359 lost=2987
Sep 10 10:01:14 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=95368316 received=95369493 lost=1177
Sep 10 10:10:15 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=95503881 received=95510899 lost=7018
Sep 10 10:20:15 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=95647326 received=95648167 lost=841
Sep 10 10:20:15 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=95656171 received=95656229 lost=58
Sep 10 10:25:15 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=95720069 received=95720301 lost=232
Sep 10 10:25:15 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=95720330 received=95720359 lost=29
Sep 10 10:26:15 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=95724883 received=95731954 lost=7071
Sep 10 10:30:15 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=95799441 received=95802283 lost=2842
Sep 10 10:36:15 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=95869972 received=95879174 lost=9202
Sep 10 10:45:15 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=96012011 received=96016390 lost=4379
Sep 10 10:50:15 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=96081435 received=96088772 lost=7337
Sep 10 10:55:15 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=96155221 received=96159774 lost=4553
Sep 10 10:55:15 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=96159803 received=96159832 lost=29
Sep 10 10:55:15 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=96159861 received=96159890 lost=29
Sep 10 10:55:15 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=96160093 received=96160209 lost=116
Sep 10 11:01:15 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=96240050 received=96243427 lost=3377
Sep 10 11:06:15 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=96309895 received=96320909 lost=11014
Sep 10 11:35:16 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=96777235 received=96777670 lost=435
Sep 10 11:35:16 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=96782020 received=96783615 lost=1595
Sep 10 11:38:16 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=96813374 received=96815033 lost=1659
Sep 10 11:55:16 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=97076192 received=97077642 lost=1450
Sep 10 11:56:16 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=97083471 received=97088569 lost=5098
Sep 10 11:57:16 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=97106199 received=97111912 lost=5713
Sep 10 12:02:16 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=97195453 received=97197077 lost=1624
Sep 10 12:13:16 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=97355159 received=97357268 lost=2109
Sep 10 12:14:16 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=97388330 received=97388359 lost=29
Sep 10 12:18:17 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=97427218 received=97439725 lost=12507
Sep 10 12:23:17 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=97509490 received=97522207 lost=12717
Sep 10 12:27:17 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=97597862 received=97600675 lost=2813
Sep 10 12:30:17 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=97642678 received=97648304 lost=5626
Sep 10 12:38:17 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=97760947 received=97772136 lost=11189
Sep 10 12:42:17 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=97840935 received=97850389 lost=9454
Sep 10 12:50:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=97983093 received=97983122 lost=29
Sep 10 12:50:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=97983180 received=97983093 lost=4294967208
Sep 10 12:50:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=97983122 received=97983180 lost=58
Sep 10 12:57:17 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=98102714 received=98105643 lost=2929
Sep 10 12:57:17 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=98105672 received=98105730 lost=58
Sep 10 13:03:17 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=98205760 received=98205789 lost=29
Sep 10 13:08:17 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=98274567 received=98275905 lost=1338
Sep 10 13:13:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=98359332 received=98359735 lost=403
Sep 10 13:18:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=98431629 received=98442995 lost=11366
Sep 10 13:22:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=98517954 received=98521376 lost=3422
Sep 10 13:22:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=98521405 received=98521434 lost=29
Sep 10 13:22:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=98522391 received=98525842 lost=3451
Sep 10 13:28:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=98597041 received=98609807 lost=12766
Sep 10 13:38:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=98776225 received=98780558 lost=4333
Sep 10 13:42:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=98850735 received=98856738 lost=6003
Sep 10 13:47:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=98943040 received=98949768 lost=6728
Sep 10 13:52:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=99021379 received=99024163 lost=2784
Sep 10 13:52:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=99026599 received=99026657 lost=58
Sep 10 13:53:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=99035154 received=99035263 lost=109
Sep 10 13:58:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=99108809 received=99121036 lost=12227
Sep 10 14:02:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=99195952 received=99202709 lost=6757
Sep 10 14:07:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=99282580 received=99291309 lost=8729
Sep 10 14:13:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=99365851 received=99374684 lost=8833
Sep 10 14:18:18 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=99457580 received=99458787 lost=1207
Sep 10 14:23:20 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=99532550 received=99544068 lost=11518
Sep 10 14:28:20 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=99615241 received=99629392 lost=14151
Sep 10 14:33:20 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=99709115 received=99716692 lost=7577
Sep 10 14:38:21 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=99792069 received=99806945 lost=14876
Sep 10 14:43:21 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=99885026 received=99895804 lost=10778
Sep 10 14:48:21 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=99976196 received=99986214 lost=10018
Sep 10 14:52:21 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=100065971 received=100071829 lost=5858
Sep 10 14:53:21 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=100084662 received=100084691 lost=29
Sep 10 14:58:21 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=100149656 received=100165024 lost=15368
Sep 10 15:02:21 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=100239399 received=100242125 lost=2726
Sep 10 15:02:21 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=100253377 received=100255088 lost=1711
Sep 10 15:07:21 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=100337384 received=100337732 lost=348
Sep 10 15:13:21 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=100431483 received=100431563 lost=80
Sep 10 15:18:21 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=100506527 received=100518196 lost=11669
Sep 10 15:18:21 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=100534436 received=100534465 lost=29
Sep 10 15:18:21 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=100534668 received=100534436 lost=4294967063
Sep 10 15:18:21 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=100534465 received=100534668 lost=203
Sep 10 15:32:22 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=100777530 received=100783997 lost=6467
Sep 10 15:37:22 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=100859260 received=100860565 lost=1305
Sep 10 15:37:22 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=100863494 received=100863987 lost=493
Sep 10 15:42:22 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=100949553 received=100952743 lost=3190
Sep 10 15:43:21 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=100960979 received=100961633 lost=654
Sep 10 15:53:22 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=101131740 received=101138961 lost=7221
Sep 10 15:57:22 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=101213503 received=101214982 lost=1479
Sep 10 15:58:22 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=101220811 received=101229099 lost=8288
Sep 10 16:02:22 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=101303696 received=101308771 lost=5075
Sep 10 16:13:22 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=101486556 received=101498787 lost=12231
Sep 10 16:28:22 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=101755056 received=101765185 lost=10129
Sep 10 16:32:22 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=101841548 received=101845927 lost=4379
Sep 10 16:33:22 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=101853280 received=101853309 lost=29
Sep 10 16:35:22 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=101895004 received=101895033 lost=29
Sep 10 16:38:22 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=101931186 received=101941023 lost=9837
Sep 10 16:42:22 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=102015807 received=102022187 lost=6380
Sep 10 16:48:22 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=102126366 received=102126395 lost=29
Sep 10 16:53:22 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=102192197 received=102199493 lost=7296
Sep 10 16:57:23 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=102270246 received=102283499 lost=13253
Sep 10 17:02:23 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=102354750 received=102366176 lost=11426
Sep 10 17:03:23 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=102368612 received=102368884 lost=272
Sep 10 17:07:23 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=102437018 received=102438439 lost=1421
Sep 10 17:08:23 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=102448212 received=102451254 lost=3042
Sep 10 17:37:23 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=102925071 received=102925477 lost=406
Sep 10 17:38:23 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=102929566 received=102936668 lost=7102
Sep 10 17:43:23 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=103002841 received=103015710 lost=12869
Sep 10 17:48:23 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=103087412 received=103094104 lost=6692
Sep 10 17:53:23 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=103162029 received=103171354 lost=9325
Sep 10 18:03:23 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=103318164 received=103324125 lost=5961
Sep 10 18:08:23 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=103391218 received=103400391 lost=9173
Sep 10 18:28:24 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=103695275 received=103702935 lost=7660
Sep 10 18:37:24 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=103844975 received=103845004 lost=29
Sep 10 18:37:24 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=103845091 received=103844975 lost=4294967179
Sep 10 18:37:24 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=103845004 received=103845091 lost=87
Sep 10 18:42:24 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=103919141 received=103919489 lost=348
Sep 10 18:53:24 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=104075925 received=104079761 lost=3836
Sep 10 19:03:24 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=104220803 received=104233745 lost=12942
Sep 10 19:23:25 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=104547082 received=104550604 lost=3522
Sep 10 19:32:25 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=104696995 received=104706420 lost=9425
Sep 11 05:20:35 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=111586680 received=111588797 lost=2117
Sep 11 06:46:36 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=112148168 received=112148197 lost=29
Sep 11 06:46:36 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=112148284 received=112148168 lost=4294967179
Sep 11 06:46:36 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=112148197 received=112148284 lost=87
Sep 11 07:51:37 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=112672227 received=112672256 lost=29
Sep 11 07:51:37 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=112672314 received=112672227 lost=4294967208
Sep 11 07:51:37 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=112672256 received=112672314 lost=58
Sep 11 09:01:38 localhost flow-capture[28588]: ftpdu_seq_check(): fail, expecting=113448924 received=113454940 lost=6016

--ZPt4rx8FFjLCG7dd--