XMODEM


XMODEM is a simple file transfer protocol developed as a quick hack by Ward Christensen for use in his 1977 MODEM.ASM terminal program. It allowed users to transmit files between their computers when both sides used MODEM. Keith Petersen made a minor update to always turn on "quiet mode", and called the result XMODEM.
XMODEM, like most file transfer protocols, breaks up the original data into a series of "packets" that are sent to the receiver, along with additional information allowing the receiver to determine whether that packet was correctly received. If an error is detected, the receiver requests that the packet be re-sent. A string of bad packets causes the transfer to abort.
XMODEM became extremely popular in the early bulletin board system market, largely because it was simple to implement. It was also fairly inefficient, and as modem speeds increased, this problem led to the development of a number of modified versions of XMODEM to improve performance or address other problems with the protocol. Christensen believed his original XMODEM to be "the single most modified program in computing history".
Chuck Forsberg collected a number of common modifications into his YMODEM protocol, but poor implementation led to a further fracturing before they were re-unified by his later ZMODEM protocol. ZMODEM became very popular, but never completely replaced XMODEM in the BBS market.

Packet structure

The original XMODEM used a 128-byte data packet, the block size used on CP/M floppy disks. The packet was prefixed by a simple 3-byte header containing a character, a "block number" from 1-255, and the "inverse" block number—255 minus the block number. Block numbering starts with 1 for the first block sent, not 0. The header was followed by the 128 bytes of data, and then a single-byte checksum. The checksum was the sum of all 128 data bytes in the packet modulo 256. The complete packet was thus 132 bytes long, containing 128 bytes of payload data, for a total channel efficiency of about 97%.
The file was marked "complete" with a character sent after the last block. This character was not in a packet, but sent alone as a single byte. Since the file length was not sent as part of the protocol, the last packet was padded out with a "known character" that could be dropped. In the original specification, this defaulted to or 26 decimal, which CP/M used as the end-of-file marker inside its own disk format. The standard suggested any character could be used for padding, but there was no way for it to be changed within the protocol itself – if an implementation changed the padding character, only clients using the same implementation would correctly interpret the new padding character.

Transfer details

Files were transferred one packet at a time. When received, the packet's checksum was calculated by the receiver and compared to the one received from the sender at the end of the packet. If the two matched, the receiver sent an message back to the sender, which then sent the next packet in sequence. If there was a problem with the checksum, the receiver instead sent a . If a was received, the sender would re-send the packet, and continued to try several times, normally ten, before aborting the transfer.
A was also sent if the receiver did not receive a valid packet within ten seconds while still expecting data due to the lack of a character. A seven-second timeout was also used within a packet, guarding against dropped connections in mid-packet.
The block numbers were also examined in a simple way to check for errors. After receiving a packet successfully, the next packet should have a one-higher number. If it instead received the same block number this was not considered serious, it was implied that the had not been received by the sender, which had then re-sent the packet. Any other packet number signalled that packets had been lost.
Transfers were receiver-driven; the transmitter would not send any data until an initial was sent by the receiver. This was a logical outcome of the way the user interacted with the sending machine, which would be remotely located. The user would navigate to the requested file on the sending machine, and then ask that machine to transfer it. Once this command was issued, the user would then execute a command in their local software to start receiving. Since the delay between asking the remote system for the file and issuing a local command to receive was unknown, XMODEM allowed up to 90 seconds for the receiver to begin issuing requests for data packets.

Problems

Although XMODEM was robust enough for a journalist in 1982 to transmit stories from Pakistan to the United States with an Osborne 1 and acoustic coupler over poor-quality telephone lines, the protocol had several flaws.

Minor problems

XMODEM was written for CP/M machines, and bears several marks of that operating system. Notably, files on CP/M were always multiples of 128 bytes, and their end was marked within a block with the character. These characteristics were transplanted directly into XMODEM. However, other operating systems did not feature either of these peculiarities, and the widespread introduction of MS-DOS in the early 1980s led to XMODEM having to be updated to notice either a or as the end-of-file marker.
For some time it was suggested that sending a character instead of an or should be supported in order to easily abort the transfer from the receiving end. Likewise, a received in place of the indicated the sender wished to cancel the transfer. However, this character could be easily "created" via simple noise-related errors of what was meant to be an or . A double- was proposed to avoid this problem, but it is not clear if this was widely implemented.

Major problems

XMODEM was designed for simplicity, without much knowledge of other file transfer protocols – which were fairly rare anyway. Due to its simplicity, there were a number of very basic errors that could cause a transfer to fail, or worse, result in an incorrect file which went unnoticed by the protocol. Most of this was due to the use of a simple checksum for error correction, which is susceptible to missing errors in the data if two bits are reversed, which can happen with a suitably short burst of noise. Additionally, similar damage to the header or checksum could lead to a failed transfer in cases where the data itself was undamaged.
Many authors introduced extensions to XMODEM to address these and other problems. Many asked for these extensions to be included as part of a new XMODEM standard. However, Ward Christensen refused to do this, as it was precisely the lack of these features, and the associated coding needed to support them, which led to XMODEM's widespread use. As he explained:

Batch transfers

Another problem with XMODEM was that it required the transfer to be user-driven rather than automated. Typically this meant the user would navigate on the sender's system to select the file they wanted, and then use a command to put that system into the "ready to send" mode. They would then trigger the transfer from their end using a command in their terminal emulator. If the user wanted to transfer another file, they would have to repeat this process again.
For automated transfers between two sites, a number of add-ons to the XMODEM protocol were implemented over time. These generally assumed the sender would continue sending file after file, with the receiver attempting to trigger the next file by sending a NAK as normal at the start of a transfer. When the NAKs timed out, it could be assumed that either there were no more files, or the link was broken anyway.

MODEM7

MODEM7, also known as MODEM7 batch or Batch XMODEM, was the first known extension of the XMODEM protocol. A normal XMODEM file transfer starts with the receiver sending a single NAK character to the sender, which then starts sending a single SOH to indicate the start of the data, and then packets of data.
MODEM7 changed this behavior only slightly, by sending the filename, in 8.3 filename format, before the . Each character was sent individually and had to be echoed by the receiver as a form of error correction. For a non-aware XMODEM implementation, this data would simply be ignored while it waited for the SOH to arrive, so the characters would not be echoed and the implementation could fall back to conventional XMODEM. With "aware" software, the file name could be used to save the file locally. Transfers could continue with another , each file is saved under the name being sent to the receiver.
Jerry Pournelle in 1983 described MODEM7 as "probably the most popular microcomputer communications program in existence".

TeLink

MODEM7 sent the filename as normal text, which meant it could be corrupted by the same problems that XMODEM was attempting to avoid. This led to the introduction of TeLink by Tom Jennings, author of the original FidoNet mailers.
TeLink avoided MODEM7's problems by standardizing a new "zero packet" containing information about the original file. This included the file's name, size, and timestamp, which were placed in a regular 128 byte XMODEM block. Whereas a normal XMODEM transfer would start with the sender sending "block 1" with a header, the TeLink header packet was labeled "block 0" and began with a . The packet contained the file creation date and time, filename up to 16 characters, the file size as a 4-byte value, and the name of the program sending the file.
A normal XMODEM implementation would simply discard the packet, the assumption being that the packet number had been corrupted. But this led to a potential time delay if the packet were discarded, as the sender could not tell whether the receiver had responded with a because it did not understand the zero packet or because there was a transmission error. As TeLink was normally used only by FidoNet software, which demanded it as part of the FidoNet standards, this did not present a real-world problem as both ends would always support this standard.
The basic "block 0" system became a standard in the FidoNet community, and was re-used by a number of future protocols like SEAlink and YMODEM.