Thursday, December 24, 2020

Changing Keyboard and Installing CDROM in DOS


Hi there. The previous four posts were about DOS and I am planning (at least) two more posts about DOS. I am writing this post to take a short break from boot sector article series. While mentioning about the usage of Norton disk editor in MBR article, I had written that who has a Norton Utilities CD or image file, must install CD drive in order to use it under DOS.

Another thing, I do, after installing DOS, is to change the keyboard layout. It is not a must of course, but I will also write a not to myself, so I don't forget how it is done. It is not something I do on daily basis, after all.


Changing Keyboard Layout in DOS
Changing the keyboard layout has very same logic in MSDOS and FreeDOS but it is done in slightly different ways. Standard ASCII character set does not contain Turkish characters such as 'ş' or 'ğ' or any Cyrillic character for example. To use these characters, it is necessary to change the charset using code pages (CPs).

The first 128 characters of the ASCII are fixed. I have not seen these changed in any standard CP. In theory, changing these characters is a problem but the 26 characters of the English alphabet is more or less common to most languages therefore they can be used as a part of localized charset. With help of CPs, the characters between 128 and 255 can be changed. I have read that some of the characters in the first half of ASCII charset had been changed in some old and incompatible CPs. I suppose, for example to print German characters with a printer that supports 7-bit ASCII, some less used characters could have been replaced with characters such as ü and ö.

In DOS, one or more CPs are concatenated in a single file and made available for users. These files have .cpi extension in MSDOS. In FreeDOS however, .cpi files are compressed with upx and have .cpx extension. Numbers are assigned to all CPs. The standard CP in IBM PC BIOS is CP 437. By the way, different OSes use different enumerations, so there may be conflicting CP numbers. The numbers, I mention here, are DOS CP numbers. FreeDOS uses CP 850 by default. For example, CP numbers 737 and 851 are in Greek, 855 is in Cyrillic, 857 in Turkish, 865 in Danish or Norwegian. A detailed list can be found in Code page article on Wikipedia.

Unfortunately, there is no easy way to find out which CP is in which file. There is a tool for reading CPI files and it includes CPI file format documentation as well. There is also a code I developed using another documentation here, but I tested it only on decompressed .cpx files of FreeDOS. There are three 256 character bitmaps in each CP. First one contains 8x16 pixel chars, second one contains 8x14 chars and last one contains 8x8 chars.

Without further ado, to use a specific charset, corresponding CP must be loaded. To do this, memory must be allocated using DISPLAY.SYS, first. This is done by DEVICE or DEVICEHIGH statements in CONFIG.SYS. Below, three pages are allocated in order to load CPs for display device CON:

DEVICE=C:\DOS\DISPLAY.SYS CON=(EGA,,3)

For printers, the same thing should be done with PRINTER.SYS. DISPLAY.SYS is not a real DOS device driver. I think, that's why FreeDOS uses DISPLAY.EXE for same purpose, instead of DISPLAY.SYS. Of course, it must be called in AUTOEXEC.BAT, since it is an executable file.

DISPLAY CON=(EGA,,3)

COUNTRY.SYS is not absolutely required for keyboard. This file contains information such as date and time formats as well as currency abbreviations of the countries. It can affect the output of DATE and TIME commands. In this file, there is another three-digit code for each country. e.g. 001 for US, 044 for UK, 049 for Germany and 090 for Turkey. Looks like, most of the codes are same as international phone codes of the countries. A detailed list is available on i8086.de. A CP number, corresponding to country code, can be given optionally as a parameter to COUNTRY.SYS. I think, if CP is not explicitly given, it finds the first CP assigned for that country. But as far as I know, COUNTRY.SYS does not load a CP. Frankly, I don't know why CP parameter is given to this file. This file must be called in CONFIG.SYS on both MSDOS and FreeDOS (by the way, FreeDOS has FDCONFIG.SYS and CONFIG.SYS is (optionally) included in it):

COUNTRY=090,857,C:\DOS\COUNTRY.SYS
COUNTRY=090,,C:\DOS\COUNTRY.SYS

And this file is under C:\FDOS\BIN in FreeDOS of course. 

Next, at least one CP from a .cpi/.cpx file must be loaded into the memory, allocated above, using MODE command with PREP parameter. Since this is a command, it must be in AUTOEXEC.BAT however it can also be called from CONFIG.SYS using INSTALL statement like any other shell command. Then with the same command, one of the CPs in memory must be selected using SEL parameter:

MODE CON CP PREP=((857) C:\DOS\EGA2.CPI)
MODE CON CP SEL=857

To load more than one CP, CP numbers must be written between parentheses separated by commas. In FreeDOS, CP 857 is located in C:\FDOS\CPI\EGA.CPX. FreeDOS can use .cpi files of MSDOS. The opposite is not possible since .cpx files are compressed. This might be possible only if they are extracted.

It can be seen that characters on the screen changes slightly after a CP is selected. After new charset is loaded and ready to print on the screen, ASCII codes that will be generated on keystrokes should have been rearranged. e.g. On Turkish keyboard, '[' should not appear when 'Ğ' is pressed. In other words, when pressing the key with 1Ah scan code, the character with ASCII code 0A7h should appear instead of 5Bh character, but at the same time, 5Bh character should appear when '8' key (with scan code 9) is pressed with AltGr. This is done by loading a new key mapping table using KEYB command and different mappings are contained in KEYBOARD.SYS or KEYBRD2.SYS files. Without any .sys file specified, KEYB command can load keyboard layouts from KEYBOARD.SYS file. Therefore, commands such as keyb gr and keyb de work without any .sys file specified. On the other hand, Turkish keyboard is located in KEYBRD2.SYS. There may be more than one keyboard layout in a language. Like Turkish F and Q keyboards. As far as I understood, when id parameter is not given to KEYB command, it loads the first (or default) layout. Default Turkish keyboard layout in FreeDOS is Q (id: 179), however in MSDOS, it is F (id: 440). Therefore, an id parameter is optional in FreeDOS however mandatory in MSDOS for Turkish Q keyboards.

KEYB TR,,C:\DOS\KEYBRD2.SYS /id:179

KEYBRD2.SYS is located under C:\FDOS\BIN in FreeDOS.

As a result of these commands, Turkish keyboard can be used. As mentioned before, keyboard would work fine without installing COUNTRY.SYS. If DISPLAY lines are missing, both of the MODE commands will fail. KEYB command runs regardless of both DISPLAY and MODE commands, but if it is run without required CPs loaded, pressing 'Ğ' will generate ° (degree character). Since 'ü', 'ç' and 'ö' characters are present in the standard CP, they will appear fine, but since there is no 'İ' (big I with dot), Shift+i will return 'ÿ' character. But I think that KEYB can be run first and CPs can be loaded afterwards.


Installing CD Drive in DOS
DOS, normally does not recognize CD drives. So, formatting journey of some users had ended at the nearest computer shop, because they had Win95 as CD and their CD drive is not recognized when booted with their system floppy. Moreover, MSDOS does not contain a CD driver. Even Win95 is installed, after it was terminated, DOS was still unable to recognize CD drive, if I remember correctly.

To install CD drive, first a driver file is required. Unfortunately, most CD-ROM drives of that time did not come with a driver floppy but most IDE/ATAPI CD drives can be simply installed with OAKCDROM.SYS. Although there are different drivers for SCSI drives, the procedure is same.

In MSDOS, the driver is first loaded with DEVICE or DEVICEHIGH statements in CONFIG.SYS. A device driver name must be given with /D: as parameter (not to be confused with drive letter, it comes later):

DEVICEHIGH=C:\CDROM\OAKCDROM.SYS /D:MSCD0001

After this line, CD driver is placed in memory with device driver named MSCD0001, but no drive letter has yet been assigned. The name of the device driver is given to MSCDEX with /D: parameter, in AUTOEXEC.BAT. In this way, first available drive letter is assigned to CD drive or another letter is assigned with /L: parameter:

MSCDEX /D:MSCD0001 /L:E

MSCDEX stands for Microsoft CD Extensions. In FreeDOS, device driver is loaded in AUTOEXEC.BAT with DEVLOAD command. The parameter /H means High Memory Area (same as DEVICEHIGH) and the parameter /Q means 'Quiet'. By the way, a driver named UDVD2.SYS is shipped with FreeDOS 1.2.

DEVLOAD /H /Q C:\FDOS\BIN\UDVD2.SYS /D:FDCD0001

And finally FreeDOS includes SHSUCDX, which is equivalent to MSCDEX.

SHSUCDX /D:FDCD0001 /L:E

UDVD2.SYS can be used in MSDOS and OAKCDROM.SYS can be used in FreeDOS as well. However, the fact that UDVD2.SYS is smaller than OAKCDROM.SYS probably means that it consumes less memory. Additionally, the loading time of UDVD2.SYS is also significantly shorter.

Wednesday, December 16, 2020

Keyboard Scan Code and Keyboard Handler


Hi there. In this article, I will explain how keyboards work.


Keyboard Scan Code
When a key is pressed on the keyboard, the keyboard does not transmit to the computer, which key is pressed such as 'N', '5' or so. If this were the case, it would be impossible to change keyboard layouts to English, Turkish or Russian etc. Instead of that, keyboard sends a key sequence number to the computer like first key pressed, eighth key pressed and so on. These key sequence numbers are called keyboard scan codes. These codes are literally the sequence numbers of the keys on the (US) keyboard. For example scan code of ESC is 1, top row keys starting from 1 to Backspace have 2 .. 14, second row starting with Tab and ending with Enter have 15 .. 28 etc. Although " (double quote) key on the Turkish keyboard is left to the 1 key, its scan code is 41 because this key is located next to Enter in the middle row on a standard AT keyboard.

Original IBM AT Keyboard (Image source: https://de.wikipedia.org/wiki/Datei:AT_keyboard.jpg)

In fact, the scan code of ESC key is 1 even though, it is located on the left side of the US AT keyboard. This is because its predecessor XT keyboard has ESC key at the top left side (like we are using today). Please note that there are no F11 and F12 keys on both keyboards. I will cover this later.


The Evolution of the Keyboard
The standards of the keyboards we use today, go back to the IBM PC's Model F keyboard which was released in 1981 to the market. IBM PC XT, which was released in March 1983, also had the same keyboard. These models don't have a specialized keyboard controller so the keyboards are not programmable. Being programmable means capabilities such as hardware self-testing, resetting of setting parameters such as key repeat frequency. This keyboards only transmit the scan codes to 8048 peripheral chip on the motherboard.
 
IBM PC AT, which was released in 1984 to the market, came with PC AT keyboard (shown in the image above). With this model, IBM changed the scan code set used in in PC XT keyboard, and added 8042 keyboard controller to the mainboard. This controller was only responsible for the keyboard and programmable features mentioned above, are added to the keyboard. Due to the backward compatibility issues, these keyboards could be operated with XT protocol thanks to a switch added to the keyboard. SysRq key is also added to this model. 

The standard 101-key keyboard (Model M) is released in April 1986. Keys such as Insert, Home are added and two new function keys F11 and F12 were also introduced.

IBM Model M keyboard (Image source: https://en.wikipedia.org/wiki/File:IBM_Model_M.png)

And finally, Microsoft added 3 more keys to the keyboards with Windows95: left and right Windows keys and right click menu key.


Make Codes, Break Codes and Different Scan Code Sets
Keyboard communicates with the computer via 60h and 64h IO ports. 60h is the data port and 64h is the command port where keyboard controller commands are sent. When a key is pressed on the keyboard, it sends the scan code of the key pressed, to the 8042 controller on the mainboard. This code is called make code. When the key is released, the break code is sent. However there are three different code standards.

Note: 60h is the port to which PS/2 mouse is connected as well. 

The first standard, called Set 1, is the standard of PC XT, I mentioned above. This is left as compatibility mode. The second standard (Set 2) comes with PC AT. 8042 controller translates set 2 codes to set 1 codes and transmits them to OS through interrupt controller. But the behavior of 8042 can also be changed. Finally, set 3 was released in October 1983 with IBM PC 3270 and this set is partially compatible with set 2. Set 3 codes are also translated into the first code set by 8042 like set 2. In short, different code sets transmitted by the keyboard, appear as just a single set on the OS side. When I connect an oscilloscope to the data pin of the keyboard, the signal I see is not the same as the value read from port 60h (unless I have an XT keyboard).

I briefly mentioned make codes of set 1 above. Break codes are obtained by doing OR operation on make codes with 80h. e.g. make code of ESC is 01h, its break code is 81h. Make code of Enter is 1Ch, break code is 9Ch. With this consideration, it can be thought that a keyboard can have a maximum of 127 keys (without zero code). But some keys generate more than one scan code. These codes are called extended scan codes beginning with 0E0h. As an example of these keys are Enter and '/' (slash) keys on the numerical keyboard. Pressing them generates 0E0h 1Ch and 0E0h 35h respectively. The break code of a key with extended code is obtained by ORing the second byte with 80h, like 0E0h 9Ch and 0E0h 0B5h. By the way, although entire 0EXh block is actually reserved for extended keys, almost all extended code keys use 0E0h prefix and in practice there are a small number of keys using 0E1h prefix.

Scan codes can be viewed in linux using showkey command with -s parameter. Eg:


To view scan codes in DOS, I wrote a simple code. First of all, VirtualBox does not recognize extended scan code set. I developed and tested the code in vmware. The code runs also find in DOSBox, but I got some strange errors while developing it in TurboC in DOSBox (this might be also because of me). However, the code runs stable in DOSBox even than vmware.
 
In the screenshot above, I pressed Enter, F10, F11 and '/' on the numerical keyboard, in given order. These codes in the output are the codes translated into first set, obviously. I terminated the program by pressing ESC. There are two keyboard controller commands in this code which I am writing to 64h port. 0ADh deactivates the keyboard, and 0AEh activates it again. In the "Evolution of the Keyboard" section, I mentioned that there were ten function keys on XT keyboards. F11 and F12 were added with ATs. That's why, while the scan codes of F1 .. F10 keys are between 3Bh and 44h, the scan code of F11 is 57h and F12's is 58h. The source code of this program is below:

#include <stdio.h>

unsigned char readport()    {
    unsigned char r;
    asm    {
        mov     dx, 0x0064   // disable keyboard
        mov     al, 0xAD
        out     dx, al
        push    dx

        mov     dx, 0x0060   // read from keyboard port
        in      al, dx
        mov     r, al

        pop     dx
        mov     al, 0xAE     // enable keyboard
        out     dx, al
    }
    return r;
}


int  main(int argc, char* argv[])    {
    unsigned char r, r0;

    for( ;; )    {
        asm { cli }     // disable interrupts
        r = readport();
        if((r ^ 0xE0) < 0x10)   {
            // if the code read is an extended code
            printf("--- Ext. Key: %02X  ", r);
            // read one more byte:
            printf("Escaped Char: %02X  \n", r0 = readport());
        }
        else if (r0 != r)       {
            // if scan code is different than the previous one
            printf("%02X\n", r);
        }

        if(r == 0x01)   {       // escape with ESC key
            asm { sti }
            break;
        }

        r0 = r;
    }

    return 0;
}


In set 2, keys pressed are represented by one-byte codes. Codes in this set are mixed order, i.e. ESC is 76h, '1' is 16h, '2' is 1Eh and so on. The scan code layout is given below: 

Image source: Wikibooks

Codes starting with 0F0h in this image are break codes of set 2. As it can be understood, the break code is represented by the same make code, preceded by 0F0h. ESC is 76h and its break code 0F0h 76h. Extended scan codes also start with 0E0h in this set. e.g. make code of left windows key is 0E0h 1Fh and its break code is 0E0h 0F0h 1Fh. 0E0h precedes the release code as in set 1.

Interestingly, there are no extended keys in set 3. Scan code layout of set 3 is viewed with this link. Detailed information about these sets and conversion table can be accessed with this link.

8042 keyboard controller normally finds the type of keyboard at the boot time and converts the scan codes coming from the keyboard. None of the present-day OSes use the first set anymore. For example, in atkdb.c keyboard driver of linux, atkbd_select_set( ... ) function return either 2 or 3. The value returned by this function is written to the element "set" in the struct "atkbd".

8042 can be configured to which set the incoming codes will be translated into. Current set is queried by writing 0F0h, 0 to port 60h. The value 43h, 41h or 3Fh returns for the first, second or third set respectively. After sending 0F0h; 1, 2 or 3 can be written to 60h port. Thus 8042 will start sending the data in the requested code set. Of course, when you do this in DOS, keyboard driver will try to interpret the codes that are belonging to a different set and the system will be unusable until it is restarted. On the other hand, showkey -s command in linux switches the keyboard to raw mode temporarily.


Communication with the Keyboard
Keyboards were connected with five pin DIN connectors until IBM PC ATs. I think, nowadays it is impossible to find these keyboards except for the flea market (or dumpsters). IBM developed PS/2 connector in 1987. There still are people who remembers that, it was a six-pin connector, purple for keyboard and green for mouse. If they are not thrown away, they can be found in cellars or warehouses. Today, almost all of them are connected over USB.

Signalling in PC/2 and DIN connectors is exactly the same. Both have Vcc and GND as well as CLK and DATA pins. The data flows synchronously (bidirectional serial synchronous transmission) with CLK signal. With each keystroke, the keyboard transmits a scan code over DATA pin synchronously with CLK signal.

Although this information is not necessary to do anything on the computer, it may be necessary to use a PC keyboard with Arduino for example or with other development boards.


What Happens When a Key is Pressed?
I already mentioned that the scan code is sent to the controller when a key is pressed on the keyboard. So, let's take a look at what happens at the CPU level.

First, the keyboard controller signals the nineteenth pin (IRQ 1 pin) of the primary 8259 interrupt controller (aka IRQ controller) and notifies processor of the interrupt request. If the processor is in 8086 mode, it calls int 09. However this IRQ must be handled by another interrupt if the processor is running in protected mode. Changing interrupt number is achieved by reprogramming the PIC.

Normally, int 09 code is loaded by BIOS. If required, OS can write a wrapper for on it or change it completely. When BIOS is done (pre-boot phase), int 09 vector points to address 0F000h:0E987h. This interrupt deactivates the keyboard temporarily and reads the scan code from port 60h. After that, Int 15h/4Fh is called. This function is normally used to change the scan code of a key (for example swapping Ctrl and Fn key)*. Next steps vary depending on the key pressed.

There is a flag in BIOS data area (BDA) for Shift, Ctrl and Alt keys at the addresses 0040h:0017h and 0018h. There are also flags for CapsLock, NumLock, Pause etc. as well.

Sources: Ralf Brown Int. List and lowlevel.eu

 
A similar flag is found at 0040h:0096h and 0097h:

Source: Ralf Brown Interrupt List

If the pressed key is one of the special keys mentioned above, int 9 arranges the related bits on above flags. For example, when right shift is pressed, scan code 36h is read and bit 0 at the address 0417h is set. When this key is released, scan code 0B6h is read and bit 0 at the address 0417h is reset by the BIOS.

When a regular key without any special purpose is pressed, it is important to distinguish whether this key is pressed alone or with a combination of another key (like Ctrl for example). Key combinations are distinguished with the flags above. Int 09 uses a table to calculate the pressed keys ASCII code. Let's assume, only the key 'A' is pressed. The scan code read is 1Eh and the corresponding ASCII code is 61h for 'a' (small letter). However, when Shift+A is pressed, although the scan code to be read is still 1Eh, the shift key flag is already active (scan code of Shift key is still read from 60h port) and with that flag active, int 09 interprets its ASCII code as 41h (capital A). Same thing happens when CapsLock is on and Shift is not pressed. When Ctrl+A is pressed, ASCII code is translated as 01h. For this purpose, the conversion table in BIOS consists of 128 lines for 128 scan codes. Columns are added to this table for a key's different combinations with Shift, with Ctrl and so on.

Scan code to ASCII translation is done for almost every key and pressed key is stored in the keyboard buffer. Since keys such as function keys, Home, PgUp etc. do not have ASCII code, their ASCII codes are stored as 0 in the keyboard buffer. Keys like Ctrl, Alt or NumLock are not stored. Keyboard buffer is located at the 1Eh offset in BDA and is 32 bytes in size. Since one scan code and one ASCII code are stored for each pressed key, up to 16 keys can be stored here. This data structure is a ring buffer. Its starting address is defined at 1Ah and ending address is at 1Ch in BDA. Since this is a ring buffer, the address in 1Ah may be greater than the address in 1Ch. When a keystroke is buffered here, the word consisting of the scan and ASCII codes of the pressed key is written to the address pointed by 1Ch and this pointer is increased by 2. If the value gets greater than 3Eh, which is the end of ring buffer, 20h is subtracted from this value. The pointer starts pointing to the address 1Eh again. Subsequently pressed key can be written to the a lower memory address than the key pressed before. When the value in 1Ch will be equal to the value in 1Ah after increase, it means that the keyboard buffer is full. The user pressed already 16 keys, BIOS queued them, but OS did/could not read them from the buffer. Some BIOSes warn the user in this case with sound from the tiny speaker on the mainboard. The value in 1Ah is decreased by two for each key read from the buffer.

So far, the scan code has been read from the port, translated into ASCII and written into the buffer. After that, the keyboard is activated again by int 09 and the interrupt ends. It returns the execution back to the program which is running before the key is pressed.

I compiled these steps I explained above from VirtualBox's source code. VBoxBiosAlternative386.asm is assembly output of VBox BIOS. In the rightmost column of the file exist the memory address of the current line of code and source code file generating that assembly command. 0F000h:E987h, the address of int 09, starts at line 18 268 in this file and its source code is at the line 974 of orgs.asm file. Deactivation of the keyboard and int 15h/4Fh call happens here. Flags for extended scan codes are arranged here (line 1011) as well. Then, int09_function function is called which is in keyboard.c file (line 371). Assembly output of this function is on line 7501 in the VBoxBiosAlternative386.asm file. This function sets or resets the related flags in a big switch/case block, when a key with special function is pressed. When a regular key is pressed, it finds the corresponding ASCII code of the key from scan_to_scanascii structure defined in keyboard.c:90. Each row in this table contains the codes of a key in case it is pressed alone, with Shift, with Ctrl and with Alt, respectively. The last element of each row indicates that the key changes its state with lock keys. 20h means with NumLock, 40h means with CapsLock. The scan-ASCII code pair is then transferred to keyboard buffer by calling enqueue_key function in line 604. This function is at line 339 in same file (and in VBoxBiosAlternative386.asm:7454 as assembly output).

In fact, at the 80h and 82h offset of BDA, there is another pair of start and end pointers for an additional keyboard buffer. This pointers were defined with AT BIOS as an alternative to the 16-key limited buffer of XT BIOS. However when ATs came out, so many programs on the market were using the old buffer hard-coded, that these new pointers became dysfunctional over time. Even VBox doesn't care about these pointers.

* When a keyboard layout is loaded in DOS, int 15h/4Fh checks whether that scan code exists in a scan code - ASCII table corresponding to new keyboard layout. If the code exists in this table, int 15h/4Fh passes the scan and ASCII codes (instead of int 09) of the key into the keyboard buffer and bypasses the execution of the rest of int 09 code by resetting carry flag on return.


Reading from Keyboard Buffer
I did not mention above any way to read from the keyboard buffer, delivering the keystrokes to a user program or OS. Accessing to the keyboard port is not an efficient method for obvious reasons, e. g. keyboard may be a non-US keyboard and controlling all those special keys like Alt, CapsLock etc. makes the code unnecessarily complicated. In addition to buffering keystrokes, BIOS also provides functions for fetching them from the buffer. These functions are provided under int 16h.

The first function is the zeroth function called while AH=00. Its job is, if there is any buffered keystroke in keyboard buffer, to return it in AX and remove it from the buffer. If the buffer is empty, the function waits until a key is pressed. Next function, called while AH=01, returns the keystroke in AX if there is any buffered key but does not remove it from the buffer. If there is no key in the buffer, unlike the zeroth function, it ends without waiting for any input. On return, it sets the zero flag if no keystroke was read.
 
These two functions above, are XT functions and compatible with XT key set. In other words, keys such as F11, F12, Home, Insert etc. that came with the AT keyboard cannot be fetched with these functions. With PC AT series, functions 10h and 11h have been added to int 16h. The job of the function 10h is the same as 00, and it also reads the keystrokes of new keys of the AT keyboards. Likewise, function 11h is compatible with function 01, too. If the function 00 is called and F12 is pressed, nothing happens, however if function 10h is called, it will return the scan code of F12.

Similarly, 20h and 21h functions are implemented for 122-key keyboards, but these functions are not available on many computers because these keyboards are not common.


A Simple Demo
I wrote a simple code below to explain what I wrote about the keyboard buffer. The program prints the BDA in an infinite loop. Since the keyboard buffer is here, the buffering of keystrokes can be clearly seen in real time. Q terminates the program and ESC clears keyboard buffer. Source code is given below:

#include<stdio.h>
#include<stdlib.h>

#define MEMLINES 10
#define MEMSIZE (MEMLINES * 16)

// return current page
unsigned char getpage()     {
    unsigned char p;
   
    asm     {
        mov     ah, 0x0F
        int     0x10
        mov     p, bh
    }

    return p;
}

// return current cursor position
void getline(unsigned char *x, unsigned char *y)    {
    unsigned char tx, ty;
    unsigned char p = getpage();

    asm     {
        mov     ah, 0x03
        mov     bh, p
        int     0x10
        mov     tx, dl
        mov     ty, dh
    }

    *x = tx;
    *y = ty;
}

// set given cursor position
void setline(unsigned char x, unsigned char y)      {
    unsigned char p = getpage();

    asm     {
        mov     ah, 0x02
        mov     bh, p
        mov     dl, x
        mov     dh, y
        int     0x10
    }
}

// listen on port 60h and return keypress and key release values
char keypress()     {
    unsigned char r;

    asm     {
        in      al, 0x60
        mov     r, al
    }

    return r;
}

// clear keyboard buffer by copying last key pointer over first
// key pointer i.e. 0040h:001Ch <-- 0040h:001Ah
void clearKBbuffer()    {
    unsigned char far *startKBbuffer = (unsigned char far *)0x0041C;
    unsigned char far *endKBbuffer = (unsigned char far *)0x0041A;

    *startKBbuffer++ = *endKBbuffer++;
    *startKBbuffer   = *endKBbuffer;

}


int main(int argc, char* argv[])        {
    int i, j;
    unsigned char curX, curY;
    // memblock points to the start of BIOS Data Area
    unsigned char far *memblock = (unsigned char far *)0x00400;

    getline(&curX, &curY);

    while(1)    {

        for(i = 0; i < MEMLINES; i++)   {
            // print offsets here:
            printf("%06X  ", memblock + (i << 4));

            // hexadecimal block is printed below:
            for(j = 0; j < 16; j++)     {
                printf("%02X ", memblock[(i << 4) + j]);
                if(j == 7)  printf("- ");
            }

            printf("  ");

            // char block is printed below:
            for(j = 0; j < 16; j++)     {
                    if((memblock[(i << 4) + j] < 32) ||
                       (memblock[(i << 4) + j] > 128))
                        printf(".");
                    else
                        printf("%c", memblock[(i << 4) +j]);
            }

            printf("\n");
        }
       
        setline(curX, curY);

        if (keypress() == 0x10)     // if Q key is pressed
            break;                  // break the infinite loop
        if (keypress() == 1)    {   // if ESC key is pressed
            clearKBbuffer();        // clear keyboard buffer
            // break;
        }
    }
   
    return 0;
}


I wrote this program in TurboC. Since TurboC can be difficult to find, its executable file can be downloaded together with its source code using this link. This program can be run in a VM as well as in DOSBox. When running in DOSBox, getline() and setline() do not work properly when the prompt is at the bottom line. It needs to be run after a cls command. The output then looks fine.

The image on the right is the screenshot, when I run the program first. The start and end pointers 1Ah and 1Ch contains 38h. Since these are equal, this means the buffer is empty. The characters I used while typing the name of the executable, still remain in the buffer. There is an 'o' character at the address 1Eh, followed by a Tab key whose ASCII code is 9 and scan code is 0Fh. While I was typing the command, I typed "pro" instead of "project" and pressed Tab for completion (FreeDOS). Then I typed, "readmem" and pressed Tab again, but this time since there are both readmem.exe and readmem.obj files in the same directory, only a dot character appeared at the prompt. Then, I pressed 'e' and Tab again and the key with scan code 1Ch, which is Enter and thus I ran the program. The scan code 1Ch is at the offset 37h, therefore the pointers are pointing at 38h.

I pressed 'wertyu' keys in turn. 'wer' is located between 38h and 3Eh which is the end of buffer. Then the ring buffer rewinds to the beginning and the remaining 'tyu' keys are buffered between 1Eh and 24h. I clear the buffer by pressing Esc (by assigning the value in 1Ah to the value in 1Ch) then I type 'asdf' (image below). By doing this, 'wer' keys are overwritten by 'asd' keys and 'f' is written to the first byte of the buffer (1Eh, overwriting 't'). Finally, I quit by pressing 'Q' without clearing the buffer. Since, I did not fetched the keys using int 16h, when the program ends, 'asdf' and 'q' keys (which are already in buffer) are read by DOS and displayed at the prompt.


The things that can be done with the keyboard are not limited to these. What is explained here, can only be an introduction to keyboards. There are dozens of controller commands, not limited to activate/deactivate keyboard. With these command, for example keyboard can be reset or hardware test can be done. As I mentioned previously, PS/2 mouse is also connected through port 60h. The only difference is that while keyboard creates IRQ1, mouse creates IRQ12. Activating or deactivating mouse is also done by sending commands to port 64h.
 

References: 
In addition to the references given in the text:


Sunday, October 25, 2020

What is the Disk System and What isn't? #3: Boot Sector


Hi there. In the previous two articles of this series, I discussed the physical structure of the disks and how they are partitioned. In this article, I will handle a more specific subject instead of a generic one. There is no boot sector in linux as we know it from DOS. NTFS on Windows is complex enough to be a topic on its own. Therefore, in this (and next) article, I will concentrate on DOS. The topic, which I will be handling in this article, will be the structure discussed in the Volume Boot Record (VBR) article on Wikipedia. At the end of this article, I explained why I chose to call this structure as "boot sector". Now, what is this boot sector really?

Yes, What is This Boot Sector?
Boot sector is the first sector of any partition with FAT and NTFS (and its predecessor HPFS) file systems (FS). This structure is not available on FSs such as ext and XFS. Its structure contains a block of code and some disk related information. Its function is similar to MBR, such that, the MBR loads the boot sector and executes it, likewise boot sector needs to continue the boot process and load the DOS kernel (IO.SYS) or the WinNT pre-kernel (NTLDR) and then NTOSKRNL.EXE. By the way, as of Windows Vista, there is /boot/bcd file instead of NTLDR [1].

Floppy disks and some USB disks cannot be partitioned because they don't have an MBR. In contrary, hard disks can be partitioned thanks to their MBR. From my previous article, we already know what BIOS and MBR code do. I also wrote repeatedly that there is no difference between MBR and boot sector from BIOS point of view (Citation: Code in the MBR and VBR is in essence loaded the same way. [2]). While the system is booting from the hard drive:
  1. BIOS reads and runs the MBR from the first sector. 
  2. MBR code reads and executes the first sector of the bootable partition.
  3. Boot sector code finds and loads IO.SYS and the boot process continues.
Second step is skipped when DOS is booting from the floppy disk (or any non-partitioned media) because the first sector of a floppy is the boot sector. And the boot sector code is loaded to 0:7C00h like MBR code.

Note: UEFI provides another method for loading OS [3]. The steps above are correct in environments without UEFI.


What is in the Boot Sector?
In the boot sector, there are data about the disk structure and the file system in that partition. Using these data, location of the (kernel) files is calculated which is required for the boot process. These data are primarily information about clusters, which are the logical unit of the file system, CHS/LBA info, disk label shown in the dir output in DOS and file allocation table (FAT) version. I will first copy the boot sectors of the systems, I installed, to files. Then, I will examine these according to the boot sector format. 

As I explained at the end of this article (under the "Difference between Boot Sector and VBR"), the terms I use for the concepts are slightly different from those on Wikipedia. The data, I mentioned above, are explained in the BIOS Parameter Block (BPB) and Design of the FAT file system articles in Wikipedia, apart from Volume Boot Record article. My personal opinion is that the term "BIOS parameter block" can be get mixed up with the term "BIOS Data Area (BDA)".


Reading the Boot Sector
I used HxD again to read the boot sector in Windows. Like I did it previously, I run HxD as administrator and opened MBR*. There were three partitions on my computers disk. I am interested in the last one with Windows installed (can be seen in the screenshot from the previous article). I can find out at which sector this partition is starting, using the information from the previous article. I wrote the UInt32 value at 1E6h offset, which is the starting address of the third partition, into the sector box above and brought up the boot sector. I am only doing this to show how much NTFS boot sector looks like FAT.

* To find partitions on GPT disks, it is necessary to read GUID entries. I will explain GPT format and how to read it later in separate article.


Red marked region is code, blue is data and green is the boot sector signature.

I had created and formatted two primary partitions on the DOS622 VM. I boot this machine with Damn Small Linux (DSL) and open the terminal. There is no need to install any program to read the boot sector in linux. I see two disks: /dev/hda1 and /dev/hda2, when I check them with sudo fdisk -l command. I can either read the disk with dd and redirect the output to hexdump, or I can read the disk with hexdump and look at the first 32 lines (512 bytes) using head command:

sudo hexdump -C -v /dev/hda1 | head -n 32

or

sudo dd if=/dev/hda1 count=1 bs=512 | hexdump -C -v

Same commands can be also applied to /dev/hda2. I actually do not need to copy these files to my machine but they can be easily copied with scp just in case, as explained in the previous article.

In FreeDOS VM, I had created one primary and one extended partition and two virtual partitions within the extended partition which I did not format. When I booted this VM with DSL and check the partitions with sudo fdisk -l, I see primary partition /dev/hda1 and extended partition /dev/hda2.

Disk /dev/hda: 1073 MB, 1073741824 bytes
64 heads, 63 sectors/track, 520 cylinders
Units = cylinders of 4032 * 512 = 2064384 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hda1   *         1        173     348736+   6  FAT16
/dev/hda2           174        520     699552    5  Extended
/dev/hda5           174        346     348736+   6  FAT16
/dev/hda6           347        520     350752+   6  FAT16


Considering their start and end sectors, /dev/hda5 and /dev/hda6 are virtual partitions in the extended partition. When I run the above commands on /dev/hda2, I see the sector that I already saw like in the previous article using the disk editor, which does have a partition table but no code. Sectors filled with 0F6h byte can be seen in /dev/hda5 and /dev/hda6 because I hadn't formatted these disks. This means that the boot sector is written only when the partition is formatted, according to the FS, it is formatted in.

But why 0F6h? This value is designed to be written on empty space when formatting Atari disks. Then IBM added it to the PC standard as it is and it has not been changed since then. As far as I know, there is no particular reason.

Finally, I will check the boot sector of a floppy disk. Therefore, I will create a 1.44MB floppy image file using dd on my linux host:

dd if=/dev/zero of=floppy.ima bs=512 count=2880

The number 2880 is the total number of sectors on a floppy disk as a product of 80 cylinders, 18 sectors and 2 heads. On Windows, a text file of this size can be created and then renamed to floppy.ima. The important thing is that the file size has to be 1 474 560 bytes.

After creating the file, I start DOS622 VM and insert the virtual floppy, I created, to the virtual drive from Settings -> Storage -> Floppy Controller. If I change to drive A: right now, I will get a General failure reading drive A error. (Please note: I came across a bug of format command in FreeDOS. Floppy disk needs to be formatted twice in FreeDOS. Changing to A: after first format, causes an error.)

format A: /S

I formatted the floppy disk with /S parameter as a system disk. So, system files (i.e. IO.SYS, MSDOS.SYS) were copied to the floppy. To look inside the floppy disk, simply open floppy.ima with hexdump. In Windows, HxD must be used, though.


Data Structures of Boot Sector
Of the structures, which are explained in BIOS parameter block article, almost only the NTFS structure is in use nowadays. There is DOS 4.0 EBPB on the VM disks and on the floppy, we just created. Let's take a look at DOS 4.0 EBPB first:

Sector OffsetSize
Description
0x003 byteJMP to the code
0x038 byteOEM NameNote1
0x0BwordBytes per sector
0x0DbyteSectors per cluster
0x0EwordReserved sectors
0x10byteNumber of FATs
0x11wordMax. num. of entries in root dirNote2
0x13wordTotal number of sectorsNote3
0x15byteMedia descriptorNote4
0x16wordSectors per FAT
0x18wordSectors per track
0x1AwordNumber of heads
0x1CdwordNumber of hidden sectors
0x20dwordTotal number of sectorsNote5
0x24bytePhysical drive numberNote6
0x25byteReservedNote7
0x26byteExtended signature (0x28 or 0x29)
0x27dwordVolume serial number
0x2B11 byteVolume label
0x368 byteFS Type
Compiled from Wikipedia. (1, 2)

Note1: OEM name field is a signature of the system formatting the disk. Normally it has no special value. However, arbitrary values might cause some version of DOS not to read the disk.
Note2: (pre FAT32) Due to the FAT FS limitations, number of entries (files + directories) under the directory "\" cannot exceed a specific limit. Root directory is special by design, it has to be in the second cluster and be fixed in size. Of course, this limitation isn't applied to sub-directories.
Note3: Since this field is 16-bit, it is deprecated for the partitions with more than 65 535 sectors. The new field at offset 0x20 takes its place. Its value is usually zero.
Note4: This byte is used to determine the type of the media. Its value is 0xF0 for hard disks and 0xF8 for 1.44M floppies. I do not think that other values will be encountered in practice.
Note5: When the field at offset 0x13 was deprecated (see Note3), this field has taken over its place. If this value is zero, too, besides 0x13, OS finds the total number of sectors from MBR.
Note6: 0x00, 0x01, ... for floppies and 0x80, 0x81, ... for hard disks.
Note7: Even though this space is reserved in documents, chkdsk command of WinNT uses this, as a flag for disk errors.

Let's examine the boot sector of a floppy disk using this information:

00000000  eb 3c 90 4d 53 44 4f 53  35 2e 30 00 02 01 01 00  |.<.MSDOS5.0.....|
00000010  02 e0 00 40 0b f0 09 00  12 00 02 00 00 00 00 00  |...@............|
00000020  00 00 00 00 00 00 29 05  14 76 1b 4e 4f 20 4e 41  |......)..v.NO NA|
00000030  4d 45 20 20 20 20 46 41  54 31 32 20 20 20 fa 33  |ME    FAT12   .3|


First three bytes contains "JMP +3Ch; NOP" commands. Since JMP itself is two bytes, the beginning of the code is at the offset 3Eh (=2+3Ch). Bytes at 3Eh (0FAh, 33h, 0C0h) is CLI; XOR AX, AX (0C0h is not shown above). Other values are:

OEM Name: MSDOS5.0
Bytes per sector: 0200h = 512 byte
Sectors per cluster: 1
Reserved sectors: 1 (the boot sector itself)
Number of FATs: 2
Max. num. of entries in root dir: 0E0h = 224
Total num. of sectors1: 0B40h = 2880
Media Descriptor: 0F0h (1.44M Floppy)
Sectors per FAT: 9
Sectors per track: 12h = 18
Number of heads: 2
Hidden sectors: 0
Total num. of sectors2: 0
Physical drive number: 0
Reserved: 0
Extended boot signature: 29h
Volume serial number: 05h, 14h, 76h, 1Bh = 1B76-1405 (same as the value in the image above)
Volume Label: NO NAME
FS Type: FAT12

Likewise, let's examine the hard disk of DOS622:

00000000  eb 3c 90 4d 53 44 4f 53  35 2e 30 00 02 04 01 00  |.<.MSDOS5.0.....|
00000010  02 00 02 00 00 f8 fa 00  3f 00 10 00 3f 00 00 00  |........?...?...|
00000020  e1 e7 03 00 80 00 29 12  b6 0b 4f 4d 53 2d 44 4f  |......)...OMS-DO|
00000030  53 5f 36 20 20 20 46 41  54 31 36 20 20 20 fa 33  |S_6   FAT16   .3|


OEM Name: MSDOS5.0
Bytes per sector: 0200h = 512 byte
Sectors per cluster: 4 (so each cluster is 2048 bytes)
Reserved sectors: 1 (the boot sector itself)
Number of FATs: 2
Max. num. of entries in root dir: 0200h =512
Total num. of sectors1: 0
Media Descriptor: 0F8h (hard disk)
Sectors per FAT: 250
Sectors per track: 3Fh = 63
Number of heads: 10h = 16
Hidden sectors: 3Fh = 63
Total num. of sectors2: 03 E7E1h = 255 969
Physical drive number: 80h
Reserved: 0
Extended boot signature: 29h
Volume serial number: 12h, 0B6h, 0Bh, 4Fh = 4F0B-B612
Volume Label: MS-DOS_6
FS Type: FAT16


And the structure of FAT32, which is referred as DOS 7.1 EBPB on Wikipedia. First 36 bytes of the structures are same:

Sector OffsetSizeDescription
0x0036 byte(see FAT16 structure)
0x24dwordSectors per FAT
0x28wordMirroring flagsNote8
0x2AwordVersion
0x2CdwordRoot directory clusterNote9
0x30wordFSINFO sectorNote10
0x32wordBackup boot sectorNote11
0x3412 byteReservedNote12
0x40bytePhysical drive numberNote6
0x41byteReservedNote7
0x42byteExtended signature (0x28 or 0x29)
0x43dwordVolume serial number
0x4711 byteVolume label
0x528 byteFS type
Compiled from Wikipedia. (1, 2)

Note8: Normally, file allocation table is kept as at least two copies. Each file read or write operation is done using both copies. With this flag, single FAT can be set as active.
Note9: This field is added in the new design, in order to remove the design limitations mentioned in Note2.
Note10: With FAT32, theoretical maximum number of clusters increased from 65526 to 228=268 435 456. Correct, 228, not 232 because 4 bits are reserved. Even in this case, there are too many clusters to examine, in order to calculate free space in FS or find an empty cluster. This causes performance issues. To overcome this, a separate data structure, called FSINFO, is defined to keep information about free space and clusters. This structure is usually kept in first sector, right after the boot sector and it has RRaA signature.
Note11: With FAT32, boot sector is backed up against any damage. This backup is always written to the sixth sector. If MBR code cannot read the zeroth sector, it tries to read the sixth sector. Although this behavior is hard-coded, a field is also defined in the primary boot sector, to point backup boot sector however it is not recommended to have a value other than six.
Not12: This field originally appears "boot file name" in the documentation. Normally, the name of the kernel file, that boot sector will load, is hard-coded in the boot code. Looks like this field is intended to keep the file name here.

I compiled some of the above notes from Microsoft's FAT32 File System Specification document. Unfortunately, this document is no longer available on MS website, but it can be downloaded here.

Finally, NTFS partitions have a very similar structure to FAT32. Like FAT32, first 36 bytes of this structure are almost the same as FAT16.

Sector OffsetSizeDescription
0x0036 byte(see FAT16 structure)
0x24bytePhysical drive numberNote6
0x25byteReserved / Unused
0x26byteExtended signature (0x80)
0x27byteReserved
0x28qwordSectors in volume
0x30qwordCluster number of Master
File Table (MFT)
0x38qwordCluster number of MFT mirror
0x40dwordClusters per File Record
Segment (FRS)Note13
0x44dwordClusters per Index BlockNote13
0x48qwordVolume serial number
0x50dwordChecksum
Compiled from Wikipedia and microsoft.com (1, 2, 3)

Note13: Two fields at the offsets 0x40 and 0x44 are actually defined as 1 byte + 3 bytes, not dwords. Three bytes reserved fields may have been intentionally defined for compatibility because these offsets has a different usage in FAT32. More precisely, to prevent a FAT32 disk utility from accidental corruption of an NTFS partition.

As I mentioned before, NTFS boot sector is similar to its predecessor, FAT but more complex. Therefore, I will explain NTFS in a separate article in order to keep this one short.

In MBR article, I mentioned two different terms about disk size. One of them was raw disk space and the other was formattable (usable) disk space. There are two fields in the tables above, i.e. reserved sectors and hidden sectors. Although they are used in some calculations, they can also be used to allocate some hidden space between root directory and boot sector, in theory. Other than that, there are two FATs that keep the location of the files on the partition. Without these structures, there is no FS at all. In other words, files cannot be written or even if they are written directly to the sectors, their beginning and end cannot be known. While raw space is the space that comes out when the number of sectors in MBR is multiplied by 512 bytes (bytes per sector), formattable space is remaining space on the disk when the above mentioned structures are taken out from the raw space.

For example, I had created a floppy image file of 1 474 560 bytes (2880 sectors). This is its raw space: I can write so many bytes on a floppy without any file structure. What does DOS say about the formatted floppy in the screenshot below?


It says, 2847 out of 2880 sectors can be used. Two FATs each of them 16 sectors plus one boot sector makes 33 sectors in total. This means, I cannot create a file larger than 1 457 664 bytes on this floppy disk.

This claim can be tested by booting with DSL and running following commands:

sudo mount /dev/fd0 /mnt
sudo rm /mnt/*
sudo dd if=/dev/zero of=/mnt/test
sudo ls -la /mnt

Remember, FreeDOS VM has three partitions. Two of them are extended and one is primary. DOS622 VM has two primary partitions and the half of the disk has no partition. Now I will demonstrate an easy way for all that is mentioned above but unfortunately only for those who have Norton Disk Editor.

I ran diskedit.exe (v2000) on the DOS622 VM and opened the partition table of the disk with Alt+A (see the next figure upper part). Afterwards, I pressed Enter again (cursor was on the first partition) and the boot sector appeared with all information read (bottom part). Moreover, the values reported by DOS, are shown in the right column. What could this be used for? Let's assume, I connect a bad hard disk or insert a bad floppy disk (which is not physically bad, of course), to my computer which is running DOS. I check the disk with the disk editor and find out, "Sectors per cluster" value in the boot sector is different from what the OS reports. By replacing this value with its correct value, I can possibly make the disk readable again - even though with a small possibility. On this screen, I can note the partition label and partition serial number down and verify them with the output of dir command after exiting the disk editor.

It is possible to open the MBR again, using Alt + A key combination in the disk editor (please note the "Absolute Sector 0" on the bottom right) and check the other partition (or partitions of FreeDOS) in the same way. Since there is no big difference between other partitions I will not examine them separately.


How Does the Boot Code Work?

a. FreeDOS Boot Code
In this section, I will examine boot code of FreeDOS and give an overview of DOS boot code as well. First of all, I should emphasize that FreeDOS boot code differs slightly on floppies (FAT12) and hard disks (mostly FAT16). In the text, I will mention the difference in the related paragraph.

When a floppy disk is formatted in FreeDOS without /S parameter, the boot code terminates the boot process abruptly with a simple error message. In other words, no boot code is written. This might be most likely because FreeDOS boot code is more complex than DOS, there may be no space for error messages like "Missing IO.SYS etc.". This code is so simple that it's not worth to dwell on it.
The best thing of FreeDOS is that it is open source. I downloaded FreeDOS boot code from github and included my own comments on some lines. These start with "; --" to distinguish easily. The original file can be downloaded from this link and the file containing my comments can be downloaded here. What this code does is not very complicated. First, a jmp command (line** 69) branches into the real code block. Right after that, there are definitions of the boot sector data structures (lines 73-90). I mentioned that this code is loaded at the address 0:7C00h. The code copies itself to address 1FE0h:0 (lines 162-168) to prevent the kernel to be loaded to address 0060h:0 from overwriting itself and continues to run from where the execution left by executing a far jmp command. Once the kernel is loaded, kernel.sys may need some data from the boot sector even if the boot code has finished running.

**: The given line numbers are with respect to the file, in which I added my comments.

There are three important values in the boot code:
1. FAT start (fat_start): The sector, from which the FAT starts. It's the sum of reserved sector and total number of sectors.

fat_start = hidden_sectors + reserved_sectors

I mentioned above that reserved sectors can be used to allocate space between boot sector and FAT. This field gives how many sectors FAT is after boot sector. For FAT12 and FAT16, this field is almost always 1, meaning FAT starts right after the boot sector. The term "hidden sectors" is kinda misleading. This field actually keeps on which sector that boot sector is. In other words, it is the LBA address of that sector. This field is only taken into account when booting. For a non-bootable disk, this field can be zero without problem.

2. Start of root directory (root_dir_start): Where the list of files in the root directory start. It is obtained by adding the product of number of FATs and sectors per FAT to the FAT start value from the first step.

root_dir_start = fat_start + number_of_FATs * sectors_per_FAT

3.First data sector (data_start): This is found by dividing the max. number of root directory entries to the 1/32 of the bytes per sector value and adding start of root directory value (from above step) to this. Since a file entry is 32 bytes in size, bytes per sector divided by 32 gives the number of file entries in a sector. And maximum number of root directory entries divided by this number gives the exact length of root directory table in sectors.

data_start = root_dir_start + root_dir_entries / (bytes_per_sector >> 5)

Note: File allocation table and structure of directory entries, in detail, are the topics of the next article.

The values above, are calculated and assigned to their variables between the lines 192 and 228. The memory addresses of these variables are 7BD2h, 7BD6h and 7BDAh, respectively. Next, entire root directory is read into the memory (lines 238 - 243). The value of DI after pop instruction in the line 240, comes originally from the value of AX pushed in the line 221. This value is the length of root directory, in sectors, when calculating the data_start value. The code between the lines 249 and 263 is searching for KERNEL.SYS file in the root directory table. If the file is found, the cluster number containing the file is pushed to the stack (line 263) and the entire FAT is read from the disk (lines 277 - 290) in size of "sectors per FAT" sectors.

Here comes the exact point, where the hard disk (FAT16) boot code differs from floppy disk (FAT12) boot code in FreeDOS. These codes probably exist in format.com as precompiled for both FAT12 and FAT16. I will explain the structure of this table and how it's read and interpreted in the next article. Basically, this table holds the consecutive cluster number or an "end of file (EOF)" mark, similar to linked list. With the help of first cluster number of KERNEL.SYS, following cluster numbers are extracted from file allocation table. The list of these clusters is written to a list at 0060h:2000h word by word (lines 301 - 327). This list is then read in a loop (lines 349 - 355) and the sectors corresponding to the clusters in the list are loaded to the address 0060h:0 and the kernel is run with the far jmp in line 353.

Note that because this code is relatively complex (and long), the only error message is simply an "Error!." (line 379). In the readDisk function, a dot char is printed on the screen for each sector read. This is probably necessary to pinpoint an unreadable sector. LBA support is checked in the readDisk function, but LBS isn't supported on floppies and this test is skipped anyway when boot media is a floppy disk (line 416). If LBA is supported, reading the disk is done using the DAP package created in an earlier phase and related LBA read function of int 13h. Otherwise, read_normal_BIOS (line 442) function is called. In this function (lines 445 - 486), the dword value, hold in LBA_SECTOR_0 and LBA_SECTOR_16 adresses is converted to CHS address. This function reads each sector to ES:63A0h and then copies it to the destination address in ES:BX (lines 493 - 500).

FreeDOS also supports FAT32 drives. In the previous article, I intentionally disabled FAT32 while installing the VM. Therefore, current disk boot sector is same as floppy boot sector, except for the FAT16 code block, mentioned above. There are two other boot sector codes named boot32.asm and boot32lb.asm for FAT32 disks. If I ever analyze them, I will do it in a separate article.


b. DOS Boot Code
DOS 6.x boot code is still under copyright and therefore there is no source code of it available but it is simpler than FreeDOS code. Even though it is under copyright, it can be disassembled in linux with following commands:

dd if=floppy.ima of=floppy.bin count=1 bs=512
objdump -M intel -D -b binary -m i8086 --adjust-vma=0x7C00 floppy.bin > floppy.asm

The variables in the boot block needs to be separated from the code. This code has to calculate the same variables to find the starting address of IO.SYS file. I don't think that this code is capable of using LBA because its simplicity. The problem with MSDOS is, it is not flexible as FreeDOS in some terms because it is running under some constraints to keep the code short but on the other hand it is very susceptible to something wrong:

  1. DOS requires the entries of kernel files (IO.SYS and MSDOS.SYS) to exist at the first and second place in the root directory listing. Even if these files are in disk, DOS boot will fail if their entries are not in first and second place. So, if I copy IO.SYS and MSDOS.SYS to a floppy or a disk manually, the boot process could fail. FreeDOS, on the other hand, allows booting even if KERNEL.SYS is copied manually as long as boot sector code searches for it.
  2. DOS also requires the IO.SYS file to be not fragmented. Most probably some sequential sectors are read instead of calculating the next cluster in which the continuation of the IO.SYS exists. If IO.SYS is fragmented, there could be no guarantee that the file read into memory is really IO.SYS. This also makes the system vulnerable to viruses.
  3. If only some parts of IO.SYS is loaded to memory by the boot sector, this leads that IO.SYS is responsible for reading the rest of itself into the memory.

How Are These Structures in Linux?
None of these exist, in short. There is neither number of sectors, heads, reserved sectors etc, nor any executable code. There are boot loaders to keep the boot chain continuous between BIOS and the kernel. If I consider GRUB, (1) GRUB Stage1 code can be written in MBR. In this case, GRUB MBR code runs core.img, finds grub.conf file and loads the selected OS. Or (2), Stage1 code is preferably in the first sector of the /boot partition. In this case, that partition has to be marked as bootable in the partition table and standard MBR code loads Stage1 code from the first sector of the partition, then core.img is loaded by Stage1 code and first step continues.

The process is similar in LILO and syslinux. By the way, since UEFI itself is a boot loader, linux kernel can be called from BIOS via UEFI without the need of any boot loader on UEFI supported machines.


Appendix: The difference between VBR and Boot Sector
The "Boot sector" article in Wikipedia covers (more generally) the structure that I previously mentioned as MBR. The hat-note in the MBR article says: "This article is about a PC-specific type of boot sector on partitioned media. For the first sector on non-partitioned media, see volume boot record.". The hat-note in the "Boot sector" article says: "This article is about the generic concept of boot sectors. For the VBR in PCs, see Volume Boot Record. For the MBR in PCs, see Master Boot Record.".

Why has there been such a naming confusion? While I was learning these topics, I never encountered the term VBR. I have always used the terms "MBR" and "boot sector" to distinguish between these two data structures, as they are in the sources which I learned these from. I could not find any VBR term in the references of VBR article itself. In short, VBR should be a relatively new term. I also checked from Michael Tischer's "PC Intern 3.0: System Programming (1st edition, 1992)". On page 557, MBR is called "partition sector". The subject, I explained in this article, is also mentioned as "Boot Sector" in this book (page 919).