Schlagwort platform.io

Angular Web App on ESP32

In progress…

TL;DR (take me to the battle field)

Introduction and Motivation

In some recent project, we added some ESP32 to replace wired communication with a BLE solution. Additionally, the customer wants to have some administrative web application to configure the device more easily and to get more detailed information in service.

Creating a simple web ab is not a hard task for an embedded C developer. But to create a beatiful or at least not shitty looking web application is just another game. When you look more closely on the available space on some ESP32, you will realize, that there is at most 16 MB of space to live with.

If you create a simple WiFi AP on some ESP32 4MB version with ESP-IDF, you can easily see, that there is not much left for BLE and APP.

Build statistics for a simple WiFi AP with only a hello world index.html on some lolin32 board

Unfortunately, I only have the 4 MB version available, which complicates the task even more. And no, I can not simply change to 16MB, since a few hundred units are already in operation, waiting for some firmware update to provide WiFi service. You know how this works… Deliver the hardware as early as possible and hand features in later ūüėõ

But this means, there is much headroom for optimization and not much space for resources and code. If you have a look at web application frameworks (e.g. react) and build some basic apps, they all come with megabytes of resources in the production build output folder. If you build a very basic hello-world angular application, you will find a few 100 kilobytes of resources after building it, to be deployed on a static web space. This is also just small enough, to be fitted in the flash.

Now, let’s talk about the solution…

Prerequisites

First of all, you will need to create an angular and an ESP-IDF project. This is easily done, if you already installed Visual Studio Code with PlatformIO.

To create an angular project, just follow the official guide. If you installed npm correctly, you can issue the commands in your Visual Studio Code Terminal, to fire up the project.

ng new esp-on-angular

After these steps, you have two separate projects. You can easily setup a script to copy the angular production build over to the ESP project, do it manually or link it in using git submodules. But this is not part of this tutorial. We simply copy over the website content manually from the angular project output folder.

In addition to npm, VSCode and PlatformIO, you could need some basic Linux system, when working on Windows, if you intend to use the automation script I’l provide later. On my machine, I use MSYS2/MinGW to get the CLI tools I need: bash, find, gzip, stat

Getting Started

Firmware Web Content Integration

While it is quite easy to simply handling incoming requests and responding to it, like you would do for some RESTful Web Service (which is the usual case for IoT stuff like the ESP32), integrating plain files is usually a more complex task. Thanks to the ESP-IDF, this is not the case.

In general, there are two options. Adding the files to a SPIFFS partition and reading it from there. Another solution is to embed the files in your firmware object. The latter one is what we will use in this tutorial, since it is more dense (in terms of size), althought it is less flexible and requires a bit more lumber in your config.

For it to work, simply add the files to board_build.embed_files environment in your platform.ini file of your ESP project. I usually put these ressource files into a separate folder called res.

board_build.embed_txtfiles = 
    res/index.html
board_build.embed_files =
    res/logo.png

When you did so, you can link the content easily with the following lines in your code, PlatformIO will care for it to be a valid symbol and to be linked in:

extern const uint8_t index_html_start[] asm("_binary_index_html_start");
extern const uint8_t index_html_end[]   asm("_binary_index_html_end");
extern const uint8_t logo_start[] asm("_binary_logo_png_start");
extern const uint8_t logo_end[]   asm("_binary_logo_png_end");

When you want to use that data in your http handler, just use the pointers declared above:

httpd_resp_set_type(httpRequest, "text/html");
httpd_resp_send(httpRequest, (const char*) 
    index_html_start, HTTPD_RESP_USE_STRLEN);

…or for your png image file…

httpd_resp_set_type(httpRequest, "image/png");
httpd_resp_send(httpRequest, (const char*) 
    logo_start, logo_start - logo_end);

I personally use only the binary version, since it can also process text files and makes the code more universal. You will later see, that there will not be much of a text left ūüôā

The symbols of your file object (the part in the asm function) is built by concatenating _binary_ with the name of your file with all special chars replaced by underscores _ and the suffix _start and _end respectively. In short: _binary_<FILENAMEREPL>_start and _binary_<FILENAMEREPL>_end.

Welp, that is quite fine, but the files still take up quite some space and we do not have a lot. When using the 4M version of the ESP32, we simply can not allow it to do so. So, what can we do further to reduce the size of our resources?

You could ask: Why not push the images and fonts to a publically accessible location in the wolrd wide web? Because some of us want to make it self contained. The reason for me have been simply the security and safety requirements of the project: Keep the system closed to the outside world, provide some WiFi Access Point and serve all files that are needed yourself. Do not depend on internet in general. If you have other requirements, kick your ESP in your local WiFi and access it from there or use the AP/Client mode, where the ESP connects to a wifi as a client, provides a WiFi AP and acts as a router between these two networks. But be warned, don’t expect too much performance from the latter option.

Welp, back to the size problem… What to do to make it fit?

Some quirks to shrink it down

Optimizing the ESP-IDF libraries and build scripts to free up some flash, is maybe quite an intensive task and needs a deep understanding of the IDF internals. So, we will first try to find some other quirks, to reduce the size of our own big blobs.

Angular – Stripping the unneeded

While angular itself is really tiny, it links in some fonts from google (Remeber, we want to be self contained). This is usually not needed, because devices nowadays have some replacement fonts installed, we can easily delete the links from the generated index.html and rely on the local resources of the client.

<link href="https://fonts.googleapis.com/css?family=Roboto:300,400,500&display=swap" rel="stylesheet">
<link href="https://fonts.googleapis.com/icon?family=Material+Icons" rel="stylesheet">

Zero Effort Zipping

While Angular already minimizes most of it’s output files, it would be really awesome to compress the resources somehow. But integrating a compression library and using it, stacks up again…

Then I had an flash of thought. Once upon a time, web speed analyzis tools started to raise warnings, when your website content was not delivered in a compressed format. Nowadays, webspace providers enable brotli and/or gzip compresson by default and there are plenty of tutorials available about WordPress and how to enable compression. All modern web browsers support gzip compression by default and offer it as a supported content encoding sheme.

Welp, but what this has to do with my web application content. Shall the ESP compress all content ahead of the transfer? Of course not! We simply compress all files ahead of linking the firmware and simply fake the content type to be the original while setting the content encoding to gzip. This way, the ESP is completely liberated from compression and we can reduce the size of our firmware as much as possible.

Do I need to compress all files manually and somehow save the original content type? Not really. I wrote a simple bash script, that does most of the work for you and creates some header with a look-up table (LUT) for the code to access it easily. A tiny URI-router will search it within the LUT and pick all information it needs from it.

Putting the pieces together

The Angular Project

The ESP Project

After we created the ESP project as described in my other post, we will start implementing a simple web service. This is done with

ESP32-EVB, PlatformIO And ESP-IDF ‚Äď Yet Another ESP32 tutorial

I already posted a tutorial on using the ESP32-EVB from olimex with Arduino. This time, I will provide the same with ESP-IDF, the original SDK from Espressif. Why I decided to do this? Because I had a project using it ūüėõ

To install PlatformIO, just follow the guide.

As shown in the below screenshots, create a new project VSCode -> The PIO-Icon -> New Project, define a project name, select Olimex ESP32-EVB with ESP-IDF framework and press Finish. After PIO downloaded all dependencies and configured your project, we are ready to go!

Now just wait until it finishes…

After PIO finished setting up the project, you will find the platformio.ini with its configuration. Add the line serial_speed = 115200 to the file.

After this, run an update on pio (especially useful if you had PlatformIO already installed).

pio update
pio upgrade --dev

We can also now build the empty application by entering pio run in the terminal.

If this succeeds, we will edit the main.c file and create some hello world application.

Hello World – Relay Toggle

Open src/main.c and edit it, so it will look like that:

#include <stdio.h>
#include "sdkconfig.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_system.h"
#include "driver/gpio.h"

#define RELAY_GPIO      32

void app_main() {

    gpio_pad_select_gpio(RELAY_GPIO); 
    /* Set the GPIO as a push/pull output */ 
    gpio_set_direction(RELAY_GPIO, GPIO_MODE_DEF_OUTPUT); 

    while(1) {
        printf("Test\r\n");
        gpio_set_level(RELAY_GPIO, 0);
        vTaskDelay(pdMS_TO_TICKS(500));
        gpio_set_level(RELAY_GPIO, 1);
        vTaskDelay(pdMS_TO_TICKS(500));
    }
}

…connect your Olimex ESP32-EVB and enter pio run -t upload in terminal.

Done! Your Relay should toogle twice every second.

The pin numbers are simply the GPIO numbers you can find in the schematics of your board, in this case, it is 32.

printf-Debugging

Despite the fact, that printf debugging is somewhat frowned upon, it is a very practical first step. And with platform.io and evaluation boards like this one, it is as easy as pressing a button. No additional wiring, no hassle with pins, ports or blown up code. We already prepared the test in our code above. It prints „Test“ every second on the serial console, just that we can not see it yet. For this to show up, you need to open the serial monitor:

From now on, you should see the printf output in your terminal every time you upload your code again:

Do not forget to close the serial monitor, when you upload your code next time. The serial is blocked by the monitor, so the uploader can not access this port.

After closing the serial monitor, you should drop back to your terminal, where you can start over with pio run -t upload.

OK, that’s all. There are many tutorials out there, how to setup a webserver, controlling pins, using SPI or bluetooth or whatever. You will find information on all peripherials here. Just remember to include the header(s) that are mentioned on top of every peripherial page, like we did with #include "driver/gpio.h" above in main.c.

Now, you have some great starting point for ESP-IDF ūüėõ

Have fun coding!

ESP32-EVB, PlatformIO And Arduino – Yet Another ESP32 tutorial

What the heck? Aren’t there enough toturials out there about ESP32? I believe: Yes, too many. And there are too many that struggle with setting up the arduino environment for non-arduino hardware. But there is a straight-forward solution that works out of the box: platform.io or also known as PIO.

And there are enought tutorials about installing platform.io. It is as easy as you can think. Install Visual Studio Code from Microsoft and install the PIO plugin in VSCode.

Then create a new project VSCode -> New Project and select Olimex ESP32-EVB with Arduino Platform. After PIO downloaded all dependencies and configured your project, we are ready to go!

Hello World – Relay Toggle

Open main.cpp and edit it, so it will look like that:

#include <Arduino.h>

const int relay1pin = 32;

void setup() {
  pinMode(relay1pin, OUTPUT);
}

void loop() {
  digitalWrite(relay1pin, HIGH);
  delay(10000);
  digitalWrite(relay1pin, LOW);
  delay(10000);
}

…connect your Olimex ESP32-EVB and hit the upload button:

Done! Your Relay should toogle every 10 seconds.

The pin numbers are simply the GPIO numbers you can find in the schematics of your board, in this case, it is 32.

printf-Debugging

Despite the fact, that printf debugging is somewhat frowned upon, it is a very practical first step. And with platform.io and evaluation boards like this one, it is as easy as pressing a button. No additional wiring, no hassle with pins, ports or blown up code.

Extend your code to look like this:

#include <Arduino.h>

const int relay1pin = 32;

void setup() {
  Serial.begin(115200);
  pinMode(relay1pin, OUTPUT);
}

void loop() {
  Serial.printf("Switch on\r\n");
  digitalWrite(relay1pin, HIGH);
  delay(10000);
  Serial.printf("Switch off\r\n");
  digitalWrite(relay1pin, LOW);
  delay(10000);
}

Add two lines to your platfromio.ini file. You will find the correct COM port in terminal when uploading the new code to your device.

In my case it is COM6 and platformio.ini looks like this:

[env:esp32-evb]
platform = espressif32
board = esp32-evb
framework = arduino
monitor_port = COM6
monitor_speed = 115200

Upload your code again after changing the ini file. If it does not print your output automatically to your terminal, start the monitor manually:

From now on, you should see the printf output in your terminal every time you upload your code again:

OK, that’s all. There are many tutorials out there, how to setup a webserver, controlling pins, using SPI or whatever. But now, you have some great arduino IDE without arduino IDE ūüėõ

Have fun coding!

Copyright © 2018 Daniel Glaser. All rights reserved

Consent Management mit Real Cookie Banner