Requesting references for learning about the upload process

Question

I would like to know of references that discuss the process of uploading code onto an Arduino board. I have read this reference on hacking the Arduino and an outline of the build process. I also found this segment on creating Arduino libraries informative.

Overall I am interested in tracing through an upload process to learn more detail about how the programming works. I would like to understand which C-libraries are used. I would like to understand how the code is uploaded onto the Arduino. One example that I would like to see is an direct upload of C or C++ code without using the IDE gui. I want to see kind of the "bare-minimum" necessary for programming the Arduino (in C/C++). I would ideally like to read about something like this which makes use of the Arduino libraries.

Looking forward to hearing back responses, and being a part of this community. Any references and comments appreciated.

Edit #1 (request for more information from the accepted answer):

How does the main sketch trace through the C/C++ code?

For example, in the make file shown in the accepted answer:

$(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(MAIN_SKETCH) -o $(MAIN_SKETCH).o
$(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)avr-libc/malloc.c -o malloc.c.o
$(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)avr-libc/realloc.c -o realloc.c.o

Does part of the main sketch transfer through in the compilation of malloc.c and realloc.c (as well as the c/c++ files)?

I would think so, being that it is compiled every time we change the main sketch.

Lastly, how is the outputted .hex file uploaded to the board?

Edit #2 Trouble compiling with make method

I am having issues with the manual compiling process, I am using Arduino 1.6.5 r5. It seems that malloc.c and realloc.c are not in the same place as they were in IDE 1.0.6.. Here is the makefile.

#
# Simple Arduino Makefile
#
# Author: Nick Gammon
# Date: 18th March 2015
#
# Edited by: Kyle Lawlor
# Date: 30th October 2015

$H# where you installed the Arduino app
ARDUINO_DIR = /home/wiz/programs/arduino-1.6.5-r5/

# various programs
CC = "$(ARDUINO_DIR)hardware/tools/avr/bin/avr-gcc"
CPP = "$(ARDUINO_DIR)hardware/tools/avr/bin/avr-g++"
AR = "$(ARDUINO_DIR)hardware/tools/avr/bin/avr-ar"
OBJ_COPY = "$(ARDUINO_DIR)hardware/tools/avr/bin/avr-objcopy"

# create a file called Blink.cpp in same base dir
MAIN_SKETCH = Blink.cpp

# compile flags for g++ and gcc

# may need to change these
F_CPU = 16000000
MCU = atmega328p

# compile flags
GENERAL_FLAGS = -c -g -Os -Wall -ffunction-sections -fdata-sections -mmcu=$(MCU) -DF_CPU=$(F_CPU)L -MMD -DUSB_VID=null -DUSB_PID=null -DARDUINO=106
CPP_FLAGS = $(GENERAL_FLAGS) -fno-exceptions
CC_FLAGS  = $(GENERAL_FLAGS)

# location of include files
INCLUDE_FILES = "-I$(ARDUINO_DIR)hardware/arduino/avr/cores/arduino" "-I$(ARDUINO_DIR)hardware/arduino/avr/variants/standard"

# library sources
LIBRARY_DIR = "$(ARDUINO_DIR)hardware/arduino/avr/cores/arduino/"

build:

    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(MAIN_SKETCH) -o $(MAIN_SKETCH).o
#   $(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)avr-libc/malloc.c -o malloc.c.o
#   $(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)avr-libc/realloc.c -o realloc.c.o
    $(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)WInterrupts.c -o WInterrupts.c.o
    $(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)wiring.c -o wiring.c.o
    $(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)wiring_analog.c -o wiring_analog.c.o
    $(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)wiring_digital.c -o wiring_digital.c.o
    $(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)wiring_pulse.c -o wiring_pulse.c.o
    $(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)wiring_shift.c -o wiring_shift.c.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)CDC.cpp -o CDC.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)HardwareSerial.cpp -o HardwareSerial.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)HID.cpp -o HID.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)IPAddress.cpp -o IPAddress.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)main.cpp -o main.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)new.cpp -o new.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)Print.cpp -o Print.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)Stream.cpp -o Stream.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)Tone.cpp -o Tone.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)USBCore.cpp -o USBCore.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)WMath.cpp -o WMath.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)WString.cpp -o WString.cpp.o
    rm core.a
#   $(AR) rcs core.a malloc.c.o
#   $(AR) rcs core.a realloc.c.o
    $(AR) rcs core.a WInterrupts.c.o
    $(AR) rcs core.a wiring.c.o
    $(AR) rcs core.a wiring_analog.c.o
    $(AR) rcs core.a wiring_digital.c.o
    $(AR) rcs core.a wiring_pulse.c.o
    $(AR) rcs core.a wiring_shift.c.o
    $(AR) rcs core.a CDC.cpp.o
    $(AR) rcs core.a HardwareSerial.cpp.o
    $(AR) rcs core.a HID.cpp.o
    $(AR) rcs core.a IPAddress.cpp.o
    $(AR) rcs core.a main.cpp.o
    $(AR) rcs core.a new.cpp.o
    $(AR) rcs core.a Print.cpp.o
    $(AR) rcs core.a Stream.cpp.o
    $(AR) rcs core.a Tone.cpp.o
    $(AR) rcs core.a USBCore.cpp.o
    $(AR) rcs core.a WMath.cpp.o
    $(AR) rcs core.a WString.cpp.o
    $(CC) -Os -Wl,--gc-sections -mmcu=$(MCU) -o $(MAIN_SKETCH).elf $(MAIN_SKETCH).o core.a -lm
    $(OBJ_COPY) -O ihex -j .eeprom --set-section-flags=.eeprom=alloc,load --no-change-warnings --change-section-lma .eeprom=0 $(MAIN_SKETCH).elf $(MAIN_SKETCH).eep
    $(OBJ_COPY) -O ihex -R .eeprom $(MAIN_SKETCH).elf $(MAIN_SKETCH).hex

The build fails unless the lines involving those two functions are commented out. I searched in the directories but had no luck in finding the libraries, so I decided to move forward in testing things out.

Here is the file I am using as Blink.cpp.

#include <Arduino.h>

void setup(){
    pinMode(13, OUTPUT);
}

void loop(){
    digitalWrite(13, HIGH);   // turn the LED on (HIGH is the voltage level)
    delay(1000);              // wait for a second
    digitalWrite(13, LOW);    // turn the LED off by making the voltage LOW
    delay(1000);              // wait for a second
}

Again, the build does compile ignoring malloc and realloc. I am unsure of whether these are critical libraries as I am not very familiar with C/C++. While the libraries do compile, there is an error derived from the delay() function in Blink.cpp.

Here is that:

"/home/wiz/programs/arduino-1.6.5-r5/hardware/tools/avr/bin/avr-gcc" -Os -Wl,--gc-sections -mmcu=atmega328p -o Blink.cpp.elf Blink.cpp.o core.a -lm
core.a(wiring.c.o): In function `delay':
/home/wiz/programs/arduino-1.6.5-r5/hardware/arduino/avr/cores/arduino/wiring.c:113: undefined reference to `yield'
collect2: error: ld returned 1 exit status
make: *** [build] Error 1

I am unsure at this point whether or not this error is related to the fact that malloc and realloc have not been compiled. I have traced back into the wiring.c file to find out where yield() was. That file starts with an #include "wiring_private.h" and here is the delay function and yield:

void delay(unsigned long ms)
{
        uint16_t start = (uint16_t)micros();

        while (ms > 0) {
                yield();
                if (((uint16_t)micros() - start) >= 1000) {
                        ms--;
                        start += 1000;
                }
        }
}

Here are the include statements from the wiring_private.h header.

#include <avr/io.h>
#include <avr/interrupt.h>
#include <stdio.h>
#include <stdarg.h>

#include "Arduino.h"

Perhaps on of these contain the yield() function, which is not being seen correctly?

Does anyone have any advice for what I am doing wrong regarding building manually with 1.6.5? Any comments or further comments will be appreciated.

Edit #3 Identified where yield() function is located.

Project directory:

.
├── abi.cpp
├── Arduino.h
├── binary.h
├── Blink.cpp
├── Blink.cpp.d
├── Blink.cpp.o
├── CDC.cpp
├── CDC.cpp.d
├── CDC.cpp.o
├── Client.h
├── core.a
├── HardwareSerial0.cpp
├── HardwareSerial1.cpp
├── HardwareSerial2.cpp
├── HardwareSerial3.cpp
├── HardwareSerial.cpp
├── HardwareSerial.cpp.d
├── HardwareSerial.cpp.o
├── HardwareSerial.h
├── HardwareSerial_private.h
├── HID.cpp
├── HID.cpp.d
├── HID.cpp.o
├── hooks.c
├── IPAddress.cpp
├── IPAddress.cpp.d
├── IPAddress.cpp.o
├── IPAddress.h
├── main.cpp
├── main.cpp.d
├── main.cpp.o
├── new.cpp
├── new.cpp.d
├── new.cpp.o
├── new.h
├── Printable.h
├── Print.cpp
├── Print.cpp.d
├── Print.cpp.o
├── Print.h
├── run.mk
├── run.mk~
├── Server.h
├── Stream.cpp
├── Stream.cpp.d
├── Stream.cpp.o
├── Stream.h
├── Tone.cpp
├── Tone.cpp.d
├── Tone.cpp.o
├── Udp.h
├── USBAPI.h
├── USBCore.cpp
├── USBCore.cpp.d
├── USBCore.cpp.o
├── USBCore.h
├── USBDesc.h
├── WCharacter.h
├── WInterrupts.c
├── WInterrupts.c.d
├── WInterrupts.c.o
├── wiring_analog.c
├── wiring_analog.c.d
├── wiring_analog.c.o
├── wiring.c
├── wiring.c.d
├── wiring.c.o
├── wiring_digital.c
├── wiring_digital.c.d
├── wiring_digital.c.o
├── wiring_private.h
├── wiring_pulse.c
├── wiring_pulse.c.d
├── wiring_pulse.c.o
├── wiring_pulse.S
├── wiring_shift.c
├── wiring_shift.c.d
├── wiring_shift.c.o
├── WMath.cpp
├── WMath.cpp.d
├── WMath.cpp.o
├── WString.cpp
├── WString.cpp.d
├── WString.cpp.o
└── WString.h

yield() is located in hooks.c. I do not understand why the function is not available to wiring.c.

Edit #4 Compilation working

Structure of arduino-1.6.5-r5/hardware/arduino/avr/cores/arduino:

.
├── abi.cpp
├── Arduino.h
├── binary.h
├── CDC.cpp
├── Client.h
├── HardwareSerial0.cpp
├── HardwareSerial1.cpp
├── HardwareSerial2.cpp
├── HardwareSerial3.cpp
├── HardwareSerial.cpp
├── HardwareSerial.h
├── HardwareSerial_private.h
├── HID.cpp
├── hooks.c
├── IPAddress.cpp
├── IPAddress.h
├── main.cpp
├── new.cpp
├── new.h
├── Printable.h
├── Print.cpp
├── Print.h
├── Server.h
├── Stream.cpp
├── Stream.h
├── Tone.cpp
├── Udp.h
├── USBAPI.h
├── USBCore.cpp
├── USBCore.h
├── USBDesc.h
├── WCharacter.h
├── WInterrupts.c
├── wiring_analog.c
├── wiring.c
├── wiring_digital.c
├── wiring_private.h
├── wiring_pulse.c
├── wiring_pulse.S
├── wiring_shift.c
├── WMath.cpp
├── WString.cpp
└── WString.h

The reason the yield() function was not recognized was because I was not compiling with 1.6.5 correctly. Below is an updated makefile that compiles the Blink.cpp seemingly correct. I have yet to verify the sketch works on upload to the Arduino (via avrdude). The mistake was that I did not compile abi.cpp, HardwareSerialx.cpp(0-3), and hooks.c. hooks.c is where the yield() is located.

Updated makefile:

#
# Simple Arduino Makefile
#
# Author: Nick Gammon
# Date: 18th March 2015
#
# Edited by: Kyle Lawlor
# Date: 30th October 2015

$H# where you installed the Arduino app
ARDUINO_DIR = /home/wiz/programs/arduino-1.6.5-r5/

# various programs
CC = "$(ARDUINO_DIR)hardware/tools/avr/bin/avr-gcc"
CPP = "$(ARDUINO_DIR)hardware/tools/avr/bin/avr-g++"
AR = "$(ARDUINO_DIR)hardware/tools/avr/bin/avr-ar"
OBJ_COPY = "$(ARDUINO_DIR)hardware/tools/avr/bin/avr-objcopy"

# create a file called Blink.cpp in same base dir
MAIN_SKETCH = Blink.cpp

# compile flags for g++ and gcc

# may need to change these
F_CPU = 16000000
MCU = atmega328p

# compile flags
GENERAL_FLAGS = -c -g -Os -Wall -ffunction-sections -fdata-sections -mmcu=$(MCU) -DF_CPU=$(F_CPU)L -MMD -DUSB_VID=null -DUSB_PID=null -DARDUINO=106
CPP_FLAGS = $(GENERAL_FLAGS) -fno-exceptions
CC_FLAGS  = $(GENERAL_FLAGS)

# location of include files
INCLUDE_FILES = "-I$(ARDUINO_DIR)hardware/arduino/avr/cores/arduino" "-I$(ARDUINO_DIR)hardware/arduino/avr/variants/standard"

# library sources
LIBRARY_DIR = "$(ARDUINO_DIR)hardware/arduino/avr/cores/arduino/"

build:

    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(MAIN_SKETCH) -o $(MAIN_SKETCH).o
#   $(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)avr-libc/malloc.c -o malloc.c.o
#   $(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)avr-libc/realloc.c -o realloc.c.o
    $(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)hooks.c -o hooks.c.o
    $(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)WInterrupts.c -o WInterrupts.c.o
    $(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)wiring.c -o wiring.c.o
    $(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)wiring_analog.c -o wiring_analog.c.o
    $(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)wiring_digital.c -o wiring_digital.c.o
    $(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)wiring_pulse.c -o wiring_pulse.c.o
    $(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)wiring_shift.c -o wiring_shift.c.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)abi.cpp -o abi.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)CDC.cpp -o CDC.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)HardwareSerial.cpp -o HardwareSerial.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)HardwareSerial0.cpp -o HardwareSerial0.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)HardwareSerial1.cpp -o HardwareSerial1.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)HardwareSerial2.cpp -o HardwareSerial2.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)HardwareSerial3.cpp -o HardwareSerial3.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)HID.cpp -o HID.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)IPAddress.cpp -o IPAddress.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)main.cpp -o main.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)new.cpp -o new.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)Print.cpp -o Print.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)Stream.cpp -o Stream.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)Tone.cpp -o Tone.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)USBCore.cpp -o USBCore.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)WMath.cpp -o WMath.cpp.o
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)WString.cpp -o WString.cpp.o
    rm core.a
#   $(AR) rcs core.a malloc.c.o
#   $(AR) rcs core.a realloc.c.o
    $(AR) rcs core.a hooks.c.o
    $(AR) rcs core.a WInterrupts.c.o
    $(AR) rcs core.a wiring.c.o
    $(AR) rcs core.a wiring_analog.c.o
    $(AR) rcs core.a wiring_digital.c.o
    $(AR) rcs core.a wiring_pulse.c.o
    $(AR) rcs core.a wiring_shift.c.o
    $(AR) rcs core.a abi.cpp.o
    $(AR) rcs core.a CDC.cpp.o
    $(AR) rcs core.a HardwareSerial.cpp.o
    $(AR) rcs core.a HardwareSerial0.cpp.o
    $(AR) rcs core.a HardwareSerial1.cpp.o
    $(AR) rcs core.a HardwareSerial2.cpp.o
    $(AR) rcs core.a HardwareSerial3.cpp.o
    $(AR) rcs core.a HID.cpp.o
    $(AR) rcs core.a IPAddress.cpp.o
    $(AR) rcs core.a main.cpp.o
    $(AR) rcs core.a new.cpp.o
    $(AR) rcs core.a Print.cpp.o
    $(AR) rcs core.a Stream.cpp.o
    $(AR) rcs core.a Tone.cpp.o
    $(AR) rcs core.a USBCore.cpp.o
    $(AR) rcs core.a WMath.cpp.o
    $(AR) rcs core.a WString.cpp.o
    $(CC) -Os -Wl,--gc-sections -mmcu=$(MCU) -o $(MAIN_SKETCH).elf $(MAIN_SKETCH).o core.a -lm
    $(OBJ_COPY) -O ihex -j .eeprom --set-section-flags=.eeprom=alloc,load --no-change-warnings --change-section-lma .eeprom=0 $(MAIN_SKETCH).elf $(MAIN_SKETCH).eep
    $(OBJ_COPY) -O ihex -R .eeprom $(MAIN_SKETCH).elf $(MAIN_SKETCH).hex

Are malloc.c and realloc.c being used anymore? I searched across 1.6.5 for a directory similar to avr-libc, I couldn't find anything. I'm guessing they are no longer in use.

Edit #5 Upload via avrdude

Basically this command worked for my board (Arduino Uno SMD):

avrdude -v -p atmega328p -c arduino -P /dev/ttyACM0 -b 115200 -D -U flash:w:/path/to/Blink.cpp.hex:i

This is all to say that the latter makefile in this post correctly builds the .hex file. Which can then successfully be uploaded onto the board. (Using Arduino 1.6.5).

score 2 · Accepted Answer · edited Jun 17 '20 at 08:21

I am interested in tracing through an upload process to learn more detail ...

I covered the uploading process in some detail here: What happens when code is uploaded using the bootloader?

I would like to understand which C-libraries are used

See my post about How the IDE organizes things

One example that I would like to see is an direct upload of C or C++ code without using the IDE gui.

I wrote a simple Makefile on the Arduino Forum:

#
# Simple Arduino Makefile
#
# Author: Nick Gammon
# Date: 18th March 2015
where you installed the Arduino app
ARDUINO_DIR = C:/Documents and Settings/Nick/Desktop/arduino-1.0.6/
various programs
CC = "$(ARDUINO_DIR)hardware/tools/avr/bin/avr-gcc"
CPP = "$(ARDUINO_DIR)hardware/tools/avr/bin/avr-g++"
AR = "$(ARDUINO_DIR)hardware/tools/avr/bin/avr-ar"
OBJ_COPY = "$(ARDUINO_DIR)hardware/tools/avr/bin/avr-objcopy"
MAIN_SKETCH = Blink.cpp
compile flags for g++ and gcc
may need to change these
F_CPU = 16000000
MCU = atmega328p
compile flags
GENERAL_FLAGS = -c -g -Os -Wall -ffunction-sections -fdata-sections -mmcu=$(MCU) -DF_CPU=$(F_CPU)L -MMD -DUSB_VID=null -DUSB_PID=null -DARDUINO=106
CPP_FLAGS = $(GENERAL_FLAGS) -fno-exceptions
CC_FLAGS  = $(GENERAL_FLAGS)
location of include files
INCLUDE_FILES = "-I$(ARDUINO_DIR)hardware/arduino/cores/arduino" "-I$(ARDUINO_DIR)hardware/arduino/variants/standard"
library sources
LIBRARY_DIR = "$(ARDUINO_DIR)hardware/arduino/cores/arduino/"
build:
$(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(MAIN_SKETCH) -o $(MAIN_SKETCH).o
$(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)avr-libc/malloc.c -o malloc.c.o
$(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)avr-libc/realloc.c -o realloc.c.o
$(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)WInterrupts.c -o WInterrupts.c.o
$(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)wiring.c -o wiring.c.o
$(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)wiring_analog.c -o wiring_analog.c.o
$(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)wiring_digital.c -o wiring_digital.c.o
$(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)wiring_pulse.c -o wiring_pulse.c.o
$(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)wiring_shift.c -o wiring_shift.c.o
$(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)CDC.cpp -o CDC.cpp.o
$(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)HardwareSerial.cpp -o HardwareSerial.cpp.o
$(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)HID.cpp -o HID.cpp.o
$(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)IPAddress.cpp -o IPAddress.cpp.o
$(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)main.cpp -o main.cpp.o
$(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)new.cpp -o new.cpp.o
$(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)Print.cpp -o Print.cpp.o
$(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)Stream.cpp -o Stream.cpp.o
$(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)Tone.cpp -o Tone.cpp.o
$(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)USBCore.cpp -o USBCore.cpp.o
$(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)WMath.cpp -o WMath.cpp.o
$(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)WString.cpp -o WString.cpp.o
rm core.a
$(AR) rcs core.a malloc.c.o
$(AR) rcs core.a realloc.c.o
$(AR) rcs core.a WInterrupts.c.o
$(AR) rcs core.a wiring.c.o
$(AR) rcs core.a wiring_analog.c.o
$(AR) rcs core.a wiring_digital.c.o
$(AR) rcs core.a wiring_pulse.c.o
$(AR) rcs core.a wiring_shift.c.o
$(AR) rcs core.a CDC.cpp.o
$(AR) rcs core.a HardwareSerial.cpp.o
$(AR) rcs core.a HID.cpp.o
$(AR) rcs core.a IPAddress.cpp.o
$(AR) rcs core.a main.cpp.o
$(AR) rcs core.a new.cpp.o
$(AR) rcs core.a Print.cpp.o
$(AR) rcs core.a Stream.cpp.o
$(AR) rcs core.a Tone.cpp.o
$(AR) rcs core.a USBCore.cpp.o
$(AR) rcs core.a WMath.cpp.o
$(AR) rcs core.a WString.cpp.o
$(CC) -Os -Wl,--gc-sections -mmcu=$(MCU) -o $(MAIN_SKETCH).elf $(MAIN_SKETCH).o core.a -lm
$(OBJ_COPY) -O ihex -j .eeprom --set-section-flags=.eeprom=alloc,load --no-change-warnings --change-section-lma .eeprom=0 $(MAIN_SKETCH).elf $(MAIN_SKETCH).eep
$(OBJ_COPY) -O ihex -R .eeprom $(MAIN_SKETCH).elf $(MAIN_SKETCH).hex

There are quite a few more complex ones around.

I want to see kind of the "bare-minimum" necessary for programming the Arduino (in C/C++).

Here:

int main ()
 {
 }

Does part of the main sketch transfer through in the compilation of malloc.c and realloc.c (as well as the c/c++ files)?

I would think so, being that it is compiled every time we change the main sketch.

I'm not sure I understand the part about "transfer through" but basically functions which are needed in the main sketch are linked into the combined file by the linker.

Effectively these are the main parts:

# Compile main sketch
    $(CPP) $(CPP_FLAGS) $(INCLUDE_FILES) $(MAIN_SKETCH) -o $(MAIN_SKETCH).o
Compile libraries
$(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)avr-libc/malloc.c -o malloc.c.o
$(CC) $(CC_FLAGS) $(INCLUDE_FILES) $(LIBRARY_DIR)avr-libc/realloc.c -o realloc.c.o

...
Remove temporary "combined libraries" file
rm core.a


Add compiled libraries into combined file "core.a"
$(AR) rcs core.a malloc.c.o
$(AR) rcs core.a realloc.c.o

...
Link main sketch to combined libraries (linker will discard things not needed)
$(CC) -Os -Wl,--gc-sections -mmcu=$(MCU) -o $(MAIN_SKETCH).elf $(MAIN_SKETCH).o core.a -lm


Make an .eep (EEPROM) file
$(OBJ_COPY) -O ihex -j .eeprom --set-section-flags=.eeprom=alloc,load --no-change-warnings --change-section-lma .eeprom=0 $(MAIN_SKETCH).elf $(MAIN_SKETCH).eep


Make the main .hex file
$(OBJ_COPY) -O ihex -R .eeprom $(MAIN_SKETCH).elf $(MAIN_SKETCH).hex

Lastly, how is the outputted .hex file uploaded to the board?

There are various ways of achieving this. The IDE calls the program avrdude which is designed to upload a .hex file to a processor. You can also just copy the .hex file to an SD card and upload it using a board programmer.

Are malloc.c and realloc.c being used anymore?

They are being used, the trick is to find them.

I can't see them being directly linked in a 1.6.5 compiler output, so I am assuming (I can't prove it right now) that they are linked in by default from libc.a (the default C library). Being a default library it is not mentioned on the linker line.

In my (Linux) installation I found that library here:

~/Development/arduino-1.6.5-r5/hardware/tools/avr/avr/lib/libc.a

If you do a nm on it you find this (amongst other things):

malloc.o:
00000002 C __brkval
         U __do_clear_bss
         U __do_copy_data
00000002 C __flp
00000146 T free
         U __heap_end
         U __heap_start
00000000 T malloc
00000000 D __malloc_heap_end
00000002 D __malloc_heap_start
00000004 D __malloc_margin
0000003e a __SP_H__
0000003d a __SP_L__
0000003f a __SREG__
00000000 a __tmp_reg__
00000001 a __zero_reg__

From man nm:

  "D"
  "d" The symbol is in the initialized data section.
 ...
  "T"
  "t" The symbol is in the text (code) section.

See http://www.linuxtopia.org/online_books/an_introduction_to_gcc/gccintro_17.html :

The C standard library itself is stored in '/usr/lib/libc.a' and contains functions specified in the ANSI/ISO C standard, such as 'printf'---this library is linked by default for every C program.

I am guessing here that malloc and free moved into the standard (default) library since version 1.0.6 (or they wanted to override it with a bugfix) and it is now being found, by default, in the standard C library as described above.

You should be able to find more about AVR libc at its page: http://www.nongnu.org/avr-libc/

Please see my edits. I am looking for a bit more information. Thank you for the answer. It is very helpful. — wgwz, Oct 27 '15 at 17:42
I am not too familiar with C terminology. Are the INCLUDE_FILES in reference to the headers? While the LIBRARY_DIR is in reference to the c and cpp file? I've looked in the directories and see those types of files. I just want to understand the distinction. I also found some assembly code (.S), is that file used in INCLUDE_FILES or LIBRARY_DIR? — wgwz, Oct 28 '15 at 17:34
The INCLUDE_FILES is the path to the .h files used in the compilation process (eg. Arduino.h). If you include other libraries (like SPI) more include files paths are added to the compile line for the main sketch. These locations may vary a bit from IDE to IDE, I think the one I quoted was form IDE 1.0.6. Assembly code would be assembled as a separate compilation unit (ie. another compile) since one compiler doesn't compile both C++ and assembler. The best way to examine this is to turn on verbose compiling, try some compiles, and see what gets generated. — Nick Gammon, Oct 28 '15 at 20:55
Hi, again. Please see the edits to my post. I hope the forums don't mind me amending the post as much as I have. — wgwz, Nov 01 '15 at 17:41
I figured that they likely were moved. Thanks for confirming this. The build process works for my setup now. The gcc reference will be helpful. Thanks, again. — wgwz, Nov 01 '15 at 21:54

Requesting references for learning about the upload process

1 Answers1

where you installed the Arduino app

various programs

compile flags for g++ and gcc

may need to change these

compile flags

location of include files

library sources

Compile libraries

Remove temporary "combined libraries" file

Add compiled libraries into combined file "core.a"

Link main sketch to combined libraries (linker will discard things not needed)

Make an .eep (EEPROM) file

Make the main .hex file