The TRUTH about KERMIT NEWS - or - What Columbia University doesn't know about File Transf

Master Index Current Directory Index Go to SkepticTank Go to Human Rights activist Keith Henson Go to Scientology cult

Skeptic Tank!

The TRUTH about KERMIT NEWS - or - What Columbia University doesn't know about File Transfers may surprise you! Last July Columbia University used its non-profit mail permit to distribute thousands of copies of Kermit News Number 5. This issue starts with an announcement that Columbia now accepts Credit Card orders. Under the heading "Kermit Saves You Money" Columbia invites businesses to switch from other vendors' shareware and commercial hardware to their product. Using a non profit mail permit to distribute this advertising gives Columbia a crucial advantage over tax paying competitors. I particularly object to Columbia's publishing incorrect and misleading information about the performance of file transfer software. That Columbia does so at public expense is an added irritant. In "The Truth about Kermit File Transfer Performance" tenured Columbia University professor Frank da Cruz proudly announces that Kermit "outperforms X-, Y-, and ZMODEM protocol transfers every time." (Kermit News July 1993 p.14) To lend credence to this incredible assertion, Frank includes results of so-called "True-Life Benchmarks". Frank's benchmarks and interpretation leave much to be desired. It is not my intention to defend a competitor's product. But, the reported Kermit efficiency of Procomm Plus (93%) and Zstem (97%) are very close to that of Columbia's own MSKermit (99%). Frank's 104% figure for MSKermit is misleading because the reduced control character prefixing responsible for the MSKermit speedup also works sending to Procomm Plus. Columbia does not disclose this inconvenient result. In fact, C-Kermit sending to Procomm Plus transferred some of Columbia's test files in less time than the C-Kermit-MSKermit combination required. After shading the truth about competitors' Kermit downloading performance, Frank's paper proceeds to redefine the YMODEM protocol I developed in the 1980's. YMODEM is a batch protocol based on XMODEM that uses 128 or 1024 byte data blocks. The sz/sx/sb 3.24 Program that Columbia used for its "tests" also supports YMODEM-g, a high speed streaming version of YMODEM. All of this is explained in the sz documentation, yet the protocol experts at Columbia University were clueless as to the actual performance potential of the sz 3.24 software they used for their True-Life Benchmarks. Thanks to Columbia's ignorance of file transfer protocols I don't know what protocol they actually used for the "XMODEM" and "YMODEM" tests listed in Columbia's True-Life Benchmarks. But Columbia's published numbers reveal a seriously slovenly experimental procedure. XMODEM and ZMODEM in all their forms transmit file data verbatim, without any prefixing. As a result, data patterns have no effect on X/YMODEM throughput. (Insert Chart 1, ys.gif, "Throughput vs File Size") This is clearly shown the chart above, where true XMODEM/YMODEM throughput is affected only by the file length. (Start-up time is less important for longer files.) (Insert Charts 2a and 2b, ks.gif and ks2.gif) Compare this with Columbia's data. These variations, which are not a function of file size, show an experimental error of more than 20 per cent. (Insert chart #3 fudge.gif) In addition, Columbia Kermit programs significantly understate the time required to perform a file transfer. Kermit reports a file transfer took 112.98 seconds when in fact the machines were tied up more than nine seconds longer. Professional-YAM's ZMODEM self reported transfer time was much more accurate, understating the actual time by less than .5 second. When this issue was raised on the Usenet comp.dcom.modems newsgroup, the frogheads suggested we ignore this Kermit overhead. To avoid such bias, the actual time should be measured with the Unix "time" command and verified with a stopwatch. One set of Columbia's measurements was so preposterous that I simply couldn't resist having a bit of fun with it. We noticed that cutting the communications speed from 19200 to 9600 actually increased the speed of a Columbia Kermit file transfer from 2648 to 2736 characters per second. (p.13, Tables 2 and 5.) This is the moral equivalent of putting a 30 mph governor on your car only to find highway travel is now quicker than when you were pushing 60. When I announced the "Columbia Stochastic Telepathic Kermit Hyperprotocol" to "explain" Columbia's numbers the Kermit faithful quite lost their sense of humor. The frogheads have flamed about tis "news release" but they have yet to explain how Columbia came up with these crazy numbers. And so we come to the The 1994 Protocol Shoot-out of Jan 6 1994 held at the Omen Technology World Corporate Headquarters. Unlike Columbia's Kermit News True-Life Benchmarks, the Shoot-out was held in public. Frank da Cruz was invited to attend, to verify the honesty and competence of the tests. As promised, we did not benchmark competitors' software while the machine was overloaded with other work. One of the tricks Columbia used to discredit ZMODEM in their "True-Life Benchmarks" was to carefully select files with redundant data that responds unusually well to Kermit's run length compression while at the same time refusing to use ZMODEM compression. The most egregious instance of this deception is seen with Columbia's selection of a Sun Sparc binary of the "uuencode" program. This is a tiny program stored in a 24576 byte file. All but a few thousand are nulls. Naturally this does wonders for Kermit transfer speeds. Had Columbia's protocol boffins read the sz 3.24 documentation they could have discovered that ZMODEM compression does even better. Kermit gets 320 per cent efficiency on this most extraordinary file; ZMODEM compression gets an even better 363 per cent efficiency. A separate compression program does even better at 646 per cent, twice as good as Kermit. Real world users download mostly compressed files, and here the winner is YMODEM-g, closely followed by MobyTurbo(Tm) ZMODEM, regular ZMODEM, with Kermit bringing up the rear. (Insert Chart # gifspeed.gif) So why is Kermit not the fastest when it only prefixes a few control characters? The question is incorrect. Columbia claims that only 0, 1, and 129 need to be prefixed, but this doesn't work sending to MSKermit 3.11. The recommendations made by Frank da Cruz do work: SET CONTROL UNPREFIX ALL SET CONTROL PREFIX 1 13 129 141 ; For sending to MS-DOS Kermit This makes C-Kermit prefix 8 characters, 0, 1, d, 7e, 81, 8d, a3, and fe (hex). Knowing this, one can understand why YMODEM-g is fastest since it does not prefix any control characters. MobyTurbo(Tm) ZMODEM is next with one control character prefixed, followed by standard ZMODEM which prefixes 5 control characters (7 when dialing out). Kermit not only prefixes 8 characters, but suffers from the startup and shutdown delays mentioned above. The most impressive demonstration, and the likely reason Frank da Cruz did not choose to appear at the Shootout, is Kermit's performance in the noise test. Or, to be more precise, Kermit's non-performance. In the Kermit News True-Life Benchmarks, Frank da Cruz specified a window size of 5 and a packet length of 5000. Undoubtedly Frank chose this long packet length to minimize Kermit's high per packet overhead compared to ZMODEM SUBpackets. XMODEM, YMODEM and Kermit cannot resend a packet with a different size. The problem is even worse with Kermit's selective retransmission protocol. When we attempted to replicate Columbia's 9600 bps plus noise test (Table 5) Kermit failed every time. Well, almost every time. The first few times Kermit worked, but that was only because C-Kermit quietly refused to send the specified 5000 byte packets because a needed "set buffer" command was not included. When Kermit actually uses the specified 5000 byte packets with noise bursts at 2 second intervals, no data packets get through. Why? It takes 5 seconds to send a 5000 byte packet at 9600 bps but the specified noise bursts come faster than that. Kermit croaks every time unless the Jim Kirk Kobiashi Maru procedure is used (relax the test). I made an informal survey of the guests at the shootout. As so many have noted, line noise today manifests itself mostly in disconnected calls. When this happens ZMODEM's Crash Recovery is relevant, allowing the safe resumption of interrupted file transfers. Since Kermit has no such feature Columbia chose not to discuss this most vital of ZMODEM's many features. What Columbia University's Kermit mavens don't know about file transfer protocols may hurt you. Chuck Forsberg WA7KGX caf@omen.COM 503-621-3406 Author of YMODEM, ZMODEM, Professional-YAM, ZCOMM, and DSZ Omen Technology Inc "The High Reliability Software" TeleGodzilla BBS:621-3746 FAX:621-3735 CIS:70007,2304 Genie:CAF 17505-V NW Sauvie IS RD Portland OR 97231


E-Mail Fredric L. Rice / The Skeptic Tank