Uploading...

Uploading is the process whereby we transfer the code we have created to the AVR device. There are two ways to do this, the normal method using the Arduino board Serial interface or the more advanced method using a programmer to write the code directly to the AVR device.

ATmel AVR devices can have more than one way of directly writing code to them such as SPI (Serial Peripheral Interface) or High Voltage. We will only deal with the more common SPI method throughout this website. Direct writing or uploading is used to load files such as bootloaders or sketch code replacing the bootloader altogether.

Flash Memory = 32768 Bytes Bootloader
Size
Sketch
Size
Reset Vector 0x7E00 512 Bytes 32256 Protected When
Lock Fuse is Set
to 0 (zero)
Reset Vector 0x7C00 1024 Bytes 31744
Reset Vector 0x7800 2048 Bytes 30720
Reset Vector 0x7000 4096 Bytes 28672
Application Area = 32768 Bytes
minus Bootloader Size
(if Used)
     
     
     
     
Application Start 0x0000      

The above shows the Flash memory allocation of the ATmega328 AVR and it is important to understand how your sketches are stored in the Flash memory before going any further.

As you can see there are 32768 Bytes of Flash available within the ATmega328 and it is split between the bootloader areas and the sketch area. With a bootloader installed, the sketches you upload to the AVR are loaded into Flash starting at address 0x0000. How big you sketch is is up to you but it cannot be larger than the total Flash size minus the bootloader block.

If the bootloader is 2048 Bytes the maximum amount of Flash available to upload a sketch into is 32768 - 2048 Bytes which leaves 30720 Bytes of Flash for your sketch. The IDE tells you this information at the end of compiling along with the amount of free SRAM. Remember, it is the bootloader code that takes care of uploading the sketch's code to Flash. The AVR's fuses have to be set correctly which is why you should never play around with them unless you fully understand how they should be set.

The bootloader sections of the Flash memory are fixed sizes as the AVR allocates "pages" to them and the bootloader can only start at the fixed locations shown as "Reset Vector" in the table. This is because the fuse settings only allow the four, fixed, addresses to be specified.

High Fuse
Bit
Bit # Reset Vector Selection Bootloader
Size
Function
BOOTSZ1 2 11 Reset Vector 0x7E00
10 Reset Vector 0x7C00
01 Reset Vector 0x7800
00 Reset Vector 0x7000
512 Bytes
1024 Bytes
2048 Bytes
4096 Bytes
Select Boot Size
BOOTSZ0 1
BOOTRST 0 1 = 0x0000, 0 = Reset Vector Selected Above   Select Reset Vector

The table above shows bits 0,1 and 2 of the High Fuse Byte where the Reset Vector and BOOTRST (Boot Reset) finctions are defined. Please note that BOOTRST controls whether or not execution starts at address 0x0000 or the Reset Vector selected with BOOTSZ0 & BOOTSZ1. We can eother have a bootloader or no bootloader as discussed above.

Bootloaders are always programmed into the AVR using a programmer which usually uses the SPI to upload the bootloader code as discussed above. The purpose of the bootloader is to check on boot (Reset of the device) to see if code is to be uploaded and if there is no code to upload, start executing the sketch code already present on the AVR device. The Arduino UNO bootloader pauses for two seconds during startup to check for code upload. If code is to be uploaded to the application area, the bootloader takes care of this by acting in a similar way to a programmer and writing the code into the application area and then executing it.

The AVR device does not need a bootloader if the code is written directly to the AVR using a programmer. This is a function of the fuse settings which will be covered in more detail later. Why do away with the bootloader?

All the other functions such as the Hardware Serial interface are unaffected but, if you don't mind the loss of Flash and the two second startup delay, leave well alone and continue to upload using the Serial interface.

Hex Files...

All code whether it is your sketch or a bootloader is transfered using the Intel HEX format. An Intel HEX file is a text file which is passed to AVRDUDE which parses the Hex file and then sends the raw binary data to the programmer. The programmer then writes the raw code onto the AVR. Remember that the bootloader is also used to write code to the application area of Flash using the Serial interface.

:107E0000112484B714BE81FFF0D085E080938100F7
:107E100082E08093C00088E18093C10086E0809377
:107E2000C20080E18093C4008EE0C9D0259A86E02C
:107E300020E33CEF91E0309385002093840096BBD3

: 10 7E00 00 112484B714BE81FFF0D085E080938100 F7

Above are the first four lines of the UNO's ATmega328 bootloader file in Intel Hex format with the first line broken into its separate elements. The format is as follows:

Start
Mark
Byte
Count
Start
Adress
Record
Type
Hex Data (AVR Instruction Code) Checksum
Byte
: 10 7E00 00 112484B714BE81FFF0D085E080938100 F7

You can see that each line of the Hex file specifies the address in Flash where the data is to be written. Now let's look at an application (sketch) file:

:100000000C945D000C9485000C9485000C94850084
:100010000C9485000C9485000C9485000C9485004C
:100020000C9485000C9485000C9485000C9485003C
:100030000C9485000C9485000C9485000C9485002C

: 10 0000 00 0C945D000C9485000C9485000C948500 84

The four Intel HEX lines above are from the Blink sketch. Notice that the Start Address is now "0000" which is the execution point for the application area of Flash.

Construction Worker

I did a count of the device signatures (unique to each variant of an ATmel ATmega/ATTiny device etc.) supported by AVRDUDE (there are many, many more types of AVR) and there are 102 variants. We will deal with just 3 here. They are, the ATTiny85, ATmega328 and the ATmega328P. I will refer to all devices as "AVR" throughout this tutorial.

Using the Arduino IDE, you have three options to upload or program your Arduino board:

The "Upload" button on the Toolbar Uses the "default" protocol set in the IDE's "preferences.txt" file or the programmer/protocol set in the IDE's "boards.txt" file if different. If the "boards.txt" definition specifies an external programmer, this will be used and will override the programmer selection in the menu.
The "Upload Using Programmer" option on the "File" menu Uses the programmer type selected from the IDE's "Tools/Programmer" list to program directly to the device/board without the use of a "bootloader". Overrides the "default" or "boards.txt" protocol. This also overwrites the bootloader, if present
The "Burn Bootloader" option on the "Tools" menu Uses the programmer selected in the IDE's "Tools/Programmer" list to "burn" a bootloader file to the device/board in question. Fuses are also set/reset using this option. If the "boards.txt" entry does not specify a bootloader, just the fuses will be set/reset.

The "bootloader" is a small program which is executed by the AVR when a Reset takes place either at power on or when the Reset button is pressed. The bootloader is used to configure the device and enable the programming of the Flash memory on the device. When you use a bootloader data is transferred via the device/board Serial Port or as supported by the bootloader and placed in Flash memory below where the bootloader is located so that you now have 2 programs loaded on the device/board.

Atmel AVRs use a memory arrangement that is known as "Harvard Architecture" as opposed to older systems which used contiguous memory layouts where each section followed on from the previous one. In early microprocessor derived computers there was usually only 1 bus over which data was moved so the CPU could only do one thing at a time i.e. it could not write to RAM while it was fetching from ROM. Data I/O was also on the same bus.

More modern computers and microprocessors are much more efficient and usually have multiple busses and CPUs that can perform multiple tasks at the same time which makes for very fast processing. The way the CPU operates has also markedly improved from the days when each instruction was fetched from ROM and executed step by step. Modern CPUs are often RISC (Reduced Instruction Set Computers) where a single program instruction results in multiple data manipulations. RISC is analogous to using an Arduino command such as "digitalWrite(13, HIGH);" where we tell pin 13 to go HIGH. What actually happens is that this command results in a series of CPU instructions being performed to complete the required function.

This is what RISC does for us at the CPU level. The illustration on the left shows the memory layout of a typical AVR with the "Program" memory at the top, "Data" memory centre and "EEPROM" memory at the bottom. The illustration also shows how the bootloader sits at the end of the Flash memory.

When a Reset occurs, the bootloader is executed first and when it has finished it points the CPU (Central Processing Unit) at the start of the Flash memory where the user's code has been placed by the bootloader. If there is no bootloader, the execution begins at the start of the Flash memory. The 2 second delay after a Reset is as a result of the bootloader. If you do away with the bootloader altogether, you not only gain the additional Flash memory where the bootloader would have been but you also do away with the Reset delay.

Flash memory (a faster, more flexible type of EEPROM) is called "non-volatile" as its content remains intact when power is removed. SRAM (Static Random Access Memory) is "volatile" memory which does not hold its data when power is removed and is therefore only used for temporary data storage.

EEPROM (Electrically Erasable Programmable Read Only Memory) is for more permanent storage of data created or modified whilst the program is running. The EEPROM data survives power loss and even SPI programming if the required fuses are correctly set.

The 3 areas of memory, Program, Data and EEPROM are separate and have their own addressing systems and data busses within the AVR which allows for fast transfer of data between them. The Data memory area also includes all of the CPU data registers as well the I/O pins, serial ports, I2C and SPI outputs.

Moving on:

There are certain circumstances where simply uploading sketches via the Arduino USB Serial port isn't enough. You might want to build a stand-alone project,  program fresh AVRs or even re-load the Arduino bootloader.

To make the AVR do what we want it to, it needs more "in-depth" programming. Unless you bought a pre-loaded AVR with a bootloader (if applicable) you are not even going to be able to upload sketches to it as the primary purpose of the bootloader is to perform the serial upload of your sketches. To make the AVR more usable we need to change/modify one or more of the following:

Most people starting out with Arduino products and projects will stick with the Serial Upload method as it requires no additional hardware or expense. But, there comes a time when Serial Upload does not fulfill all needs. AVR programming requires an ISP (In System Programmer) which allows you to program all aspects of the AVR directly. You can use a dedicated ISP or an Arduino board running an ISP sketch (ArduinoISP).

A dedicated ISP such as the AVRISP MKII An Arduino board running an ISP sketch
Pros:
Specifically designed for In System Programming
Various types and manufacturers to choose from

Cons:
Can be expensive for intermittent use
Not re-usable for other jobs
Can be difficult to set up
Pros:
Can be cheaper than a dedicated ISP
Can be used for other jobs when not being used as an ISP
Software can be easily updated or adapted by the user
Can offer more functions than some dedicated ISPs

Cons:
None really

The AVRISP MKII In System Programmer is shown here in a form made by Olimex. There are quite a few other variants and it's up to you which you choose if a dedicated ISP is your choice.

Using an Arduino board as an ISP only requires uploading a special sketch called "ArduinoISP", which comes with the Arduino IDE, to the board, mounting the target device on a breadboard and making a few connections using jumper wires. Once you have finished programming, your Arduino is ready for another job. You could even make a Shield with a Zero Insertion Force (ZIF) socket for serious programming.

Another consideration is that an Arduino board using other versions of the ArduinoISP sketch can provide a clock signal which is useful for devices who's initial state is unknown or help with "unbricking" devices where programming has failed for some reason.

ISPs use their own set of signals using a 6 or 10 pin connector (now you know what the 6 pins at the end of the Uno board are for) or, in the case of the Arduino ISP, jumper leads.

If you only intend working with ATmega328P devices as used on the Arduino Uno for example, you will have an easy life. To program a new device you will only need an ISP, a breadboard and some jumper wires. Everything else is provided for within the Arduino IDE. If you want to use more exotic and perhaps less compatible devices such as the ATmega328 or ATTiny85, things start to get a bit harder I'm afraid but all will be explained later.

Whatever you choose to do, there is always a solution but, be careful! There is an awful lot of mis-information and, in fact, totally wrong information being bandied about on Internet Forums. Do your research carefully and always check out anything you read before trying it.

Finally, try to burn your own bootloaders as AVRs without bootloaders installed are a lot cheaper than ones with a bootloader and there's a lot of satisfaction in doing the job yourself. These tutorials will, hopefully, help you to become proficient at defining AVR types and programming them successfully.