Supporting ETag http headers on static files (#7687)

* Supporting ETag http headers on static files

* Supporting ETag http headers on static files

* WebServer Example and Doku

* new template for readme added.

* example updated, som more TRACE output.

* better TRACE formatting.

* upload and trace enhancements

* Create .skip.esp32h2

* Update libraries/WebServer/examples/WebServer/data/index.htm
Co-authored-by: Lucas Saavedra Vaz <lucassvaz@yahoo.com.br>

* files.htm trailing newLine

---------
Co-authored-by: Jan Procházka <90197375+P-R-O-C-H-Y@users.noreply.github.com>
Co-authored-by: Lucas Saavedra Vaz <lucassvaz@yahoo.com.br>

libraries/WebServer/examples/WebServer/.skip.esp32h2

+

libraries/WebServer/examples/WebServer/README.md

+# Arduino-ESP32 WebServer Example for WebServer Library
+
+This example shows different techniques on how to use and extend the WebServer for specific purposes
+
+It is a small project in it's own and has some files to use on the web server to show how to use simple REST based services.
+
+This example requires some space for a filesystem and runs fine boards with 4 MByte flash using the following options:
+
+* Board: ESP32 Dev Module
+* Partition Scheme: Default 4MB with spiffs (1.2MB APP/1.5MB SPIFFS)
+    but LittleFS will be used in the partition (not SPIFFS)
+
+It features
+
+* Setup a web server
+* redirect when accessing the url with servername only
+* get real time by using builtin NTP functionality
+* send HTML responses from Sketch (see builtinfiles.h)
+* use a LittleFS file system on the data partition for static files
+* use http ETag Header for client side caching of static files
+* use custom ETag calculation for static files
+* extended FileServerHandler for uploading and deleting static files
+* uploading files using drag & drop
+* serve APIs using REST services (/api/list, /api/sysinfo)
+* define HTML response when no file/api/handler was found
+
+## Supported Targets
+
+Currently, this example supports the following targets.
+
+| Supported Targets | ESP32 | ESP32-S2 | ESP32-C3 |
+| ----------------- | ----- | -------- | -------- |
+|                   | yes   | yes      | yes      |
+
+## Use the Example
+
+How to install the Arduino IDE: [Install Arduino IDE](https://github.com/espressif/arduino-esp32/tree/master/docs/arduino-ide).
+
+* In the file `secrets.h` you can add the home WiFi network name ans password.
+* Compile and upload to the device.
+* Have a look into the monitoring output.
+* Open <http://webserver> or <http://(ip-address)> using a browser.
+* You will be redirected to <http://webserver/$upload.htm> as there are no files yet in the file system.
+* Drag the files from the data folder onto the drop area shown in the browser.
+* See below for more details
+
+## Implementing a web server
+
+The WebServer library offers a simple path to implement a web server on a ESP32 based board.
+
+The advantage on using the WebServer instead of the plain simple WiFiServer is that the WebServer
+takes much care about the http protocol conventions and features and allows easily access to parameters.
+It offers plug-in capabilities by registering specific functionalities that will be outlined below.
+
+### Initialization
+
+In the setup() function in the webserver.ino sketch file the following steps are implemented to make the webserver available on the local network.
+
+* Create a webserver listening to port 80 for http requests.
+* Initialize the access to the filesystem in the free flash memory.
+* Connect to the local WiFi network. Here is only a straight-forward implementation hard-coding network name and passphrase. You may consider to use something like the WiFiManager library in real applications.
+* Register the device in DNS using a known hostname.
+* Registering several plug-ins (see below).
+* Starting the web server.
+
+### Running
+
+In the loop() function the web server will be given time to receive and send network packages by calling
+`server.handleClient();`.
+
+## Registering simple functions to implement RESTful services
+
+Registering function is the simplest integration mechanism available to add functionality. The server offers the `on(path, function)` methods that take the URL and the function as parameters.
+
+There are 2 functions implemented that get registered to handle incoming GET requests for given URLs.
+
+The JSON data format is used often for such services as it is the "natural" data format of the browser using javascript.
+
+When the **handleSysInfo()** function is registered and a browser requests for <http://webserver/api/sysinfo> the function will be called and can collect the requested information.
+
+> ```CPP
+> server.on("/api/sysinfo", handleSysInfo);
+> ```
+
+The result in this case is a JSON object that is assembled in the result String variable and the returned as a response to the client also giving the information about the data format.
+
+You can try this request in a browser by opening <http://webserver/api/sysinfo> in the address bar.
+
+> ```CPP
+> server.on("/api/sysinfo", handleList);
+> ```
+
+The function **handleList()** is registered the same way to return the list of files in the file system also returning a JSON object including name, size and the last modification timestamp.
+
+You can try this request in a browser by opening <http://webserver/api/list> in the address bar.
+
+## Registering a function to send out some static content from a String
+
+This is an example of registering a inline function in the web server.
+The 2. parameter of the on() method is a so called CPP lamda function (without a name)
+that actually has only one line of functionality by sending a string as result to the client.
+
+> ``` cpp
+> server.on("/$upload.htm", []() {
+>   server.send(200, "text/html", FPSTR(uploadContent));
+> });
+> ```
+
+Here the text from a static String with html code is returned instead of a file from the filesystem.
+The content of this string can be found in the file `builtinfiles.h`. It contains a small html+javascript implementation
+that allows uploading new files into the empty filesystem.
+
+Just open <http://webserver/$upload.htm> and drag some files from the data folder on the drop area.
+
+## Registering a function to handle requests to the server without a path
+
+Often servers are addressed by using the base URL like <http://webserver/> where no further path details is given.
+Of course we like the user to be redirected to something usable. Therefore the `handleRoot()` function is registered:
+
+> ``` cpp
+> server.on("/$upload.htm", handleRoot);
+> ```
+
+The `handleRoot()` function checks the filesystem for the file named **/index.htm** and creates a redirect to this file when the file exists.
+Otherwise the redirection goes to the built-in **/$upload.htm** web page.
+
+## Using the serveStatic plug-in
+
+The **serveStatic** plug in is part of the library and handles delivering files from the filesystem to the client. It can be customized in some ways.
+
+> ``` cpp
+> server.enableCORS(true);
+> server.enableETag(true);
+> server.serveStatic("/", LittleFS, "/");
+> ```
+
+### Cross-Origin Ressource Sharing (CORS)
+
+The `enableCORS(true)` function adds a `Access-Control-Allow-Origin: *` http-header to all responses to the client
+to inform that it is allowed to call URLs and services on this server from other web sites.
+
+The feature is disabled by default (in the current version) and when you like to disable this then you should call `enableCORS(false)` during setup.
+
+* Web sites providing high sensitive information like online banking this is disabled most of the times.
+* Web sites providing advertising information or reusable scripts / images this is enabled.
+
+### enabling ETag support
+
+To enable this in the embedded web server the `enableETag()` can be used.
+(next to enableCORS)
+
+In the simplest version just call `enableETag(true)` to enable the internal ETag generation that calcs the hint using a md5 checksum in base64 encoded form. This is an simple approach that adds some time for calculation on every request but avoids network traffic.
+
+The headers will look like:
+
+``` txt
+If-None-Match: "GhZka3HevoaEBbtQOgOqlA=="
+ETag: "GhZka3HevoaEBbtQOgOqlA=="
+```
+
+
+### ETag support customization
+
+The enableETag() function has an optional second optional parameter to provide a function for ETag calculation of files.
+
+The function enables eTags for all files by using calculating a value from the last write timestamp:
+
+``` cpp
+server.enableETag(true, [](FS &fs, const String &path) -> String {
+  File f = fs.open(path, "r");
+  String eTag = String(f.getLastWrite(), 16);  // use file modification timestamp to create ETag
+  f.close();
+  return (eTag);
+});
+```
+
+The headers will look like:
+
+``` txt
+ETag: "63bbaeb5"
+If-None-Match: "63bbaeb5"
+```
+
+
+## Registering a full-featured handler as plug-in
+
+The example also implements the class `FileServerHandler` derived from the class `RequestHandler` to plug in functionality
+that can handle more complex requests without giving a fixed URL.
+It implements uploading and deleting files in the file system that is not implemented by the standard server.serveStatic functionality.
+
+This class has to implements several functions and works in a more detailed way:
+
+* The `canHandle()` method can inspect the given http method and url to decide weather the RequestFileHandler can handle the incoming request or not.
+
+  In this case the RequestFileHandler will return true when the request method is an POST for upload or a DELETE for deleting files.
+
+  The regular GET requests will be ignored and therefore handled by the also registered server.serveStatic handler.
+
+* The function `handle()` then implements the real deletion of the file.
+
+* The `canUpload()`and `upload()` methods work similar while the `upload()` method is called multiple times to create, append data and close the new file.
+
+## File upload
+
+By opening <http://webserver/$upload.htm> you can easily upload files by dragging them over the drop area.
+
+Just take the files from the data folder to create some files that can explore the server functionality.
+
+Files will be uploaded to the root folder of the file system. and you will see it next time using  <http://webserver/files.htm>.
+
+The filesize that is uploaded is not known when the upload mechanism in function
+FileServerHandler::upload gets started.
+
+Uploading a file that fits into the available filesystem space
+can be found in the Serial output:
+
+``` txt
+starting upload file /file.txt...
+finished.
+1652 bytes uploaded.
+```
+
+Uploading a file that doesn't fit can be detected while uploading when writing to the filesystem fails.
+However upload cannot be aborted by the current handler implementation.
+
+The solution implemented here is to delete the partially uploaded file and wait for the upload ending.
+The following can be found in the Serial output:
+
+``` txt
+starting upload file /huge.jpg...
+./components/esp_littlefs/src/littlefs/lfs.c:584:error: No more free space 531
+  write error!
+finished.
+```
+
+You can see on the Serial output that one filesystem write error is reported.
+
+Please be patient and wait for the upload ending even when writing to the filesystem is disabled
+it maybe take more than a minute.
+
+## Registering a special handler for "file not found"
+
+Any other incoming request that was not handled by the registered plug-ins above can be detected by registering
+
+> ``` cpp
+> // handle cases when file is not found
+> server.onNotFound([]() {
+>   // standard not found in browser.
+>   server.send(404, "text/html", FPSTR(notFoundContent));
+> });
+> ```
+
+This allows sending back an "friendly" result for the browser. Here a simple html page is created from a static string.
+You can easily change the html code in the file `builtinfiles.h`.
+
+## customizations
+
+You may like to change the hostname and the timezone in the lines:
+
+> ``` cpp
+> #define HOSTNAME "webserver"
+> #define TIMEZONE "CET-1CEST,M3.5.0,M10.5.0/3"
+> ```
+
+## Troubleshooting
+
+Have a look in the Serial output for some additional runtime information.
+
+## Contribute
+
+To know how to contribute to this project, see [How to contribute.](https://github.com/espressif/arduino-esp32/blob/master/CONTRIBUTING.rst)
+
+If you have any **feedback** or **issue** to report on this example/library, please open an issue or fix it by creating a new PR. Contributions are more than welcome!
+
+Before creating a new issue, be sure to try Troubleshooting and check if the same issue was already created by someone else.
+
+## Resources
+
+* Official ESP32 Forum: [Link](https://esp32.com)
+* Arduino-ESP32 Official Repository: [espressif/arduino-esp32](https://github.com/espressif/arduino-esp32)
+* ESP32 Datasheet: [Link to datasheet](https://www.espressif.com/sites/default/files/documentation/esp32_datasheet_en.pdf)
+* ESP32-S2 Datasheet: [Link to datasheet](https://www.espressif.com/sites/default/files/documentation/esp32-s2_datasheet_en.pdf)
+* ESP32-C3 Datasheet: [Link to datasheet](https://www.espressif.com/sites/default/files/documentation/esp32-c3_datasheet_en.pdf)
+* Official ESP-IDF documentation: [ESP-IDF](https://idf.espressif.com)

libraries/WebServer/examples/WebServer/WebServer.ino

+// @file WebServer.ino
+// @brief Example WebServer implementation using the ESP32 WebServer
+// and most common use cases related to web servers.
+//
+// * Setup a web server
+// * redirect when accessing the url with servername only
+// * get real time by using builtin NTP functionality
+// * send HTML responses from Sketch (see builtinfiles.h)
+// * use a LittleFS file system on the data partition for static files
+// * use http ETag Header for client side caching of static files
+// * use custom ETag calculation for static files
+// * extended FileServerHandler for uploading and deleting static files
+// * extended FileServerHandler for uploading and deleting static files
+// * serve APIs using REST services (/api/list, /api/sysinfo)
+// * define HTML response when no file/api/handler was found
+//
+// See also README.md for instructions and hints.
+//
+// Please use the following Arduino IDE configuration
+//
+// * Board: ESP32 Dev Module
+// * Partition Scheme: Default 4MB with spiffs (1.2MB APP/1.5MB SPIFFS)
+//     but LittleFS will be used in the partition (not SPIFFS)
+// * other setting as applicable
+//
+// Changelog:
+// 21.07.2021 creation, first version
+// 08.01.2023 ESP32 version with ETag
+
+#include <Arduino.h>
+#include <WebServer.h>
+
+#include "secrets.h"  // add WLAN Credentials in here.
+
+#include <FS.h>        // File System for Web Server Files
+#include <LittleFS.h>  // This file system is used.
+
+// mark parameters not used in example
+#define UNUSED __attribute__((unused))
+
+// TRACE output simplified, can be deactivated here
+#define TRACE(...) Serial.printf(__VA_ARGS__)
+
+// name of the server. You reach it using http://webserver
+#define HOSTNAME "webserver"
+
+// local time zone definition (Berlin)
+#define TIMEZONE "CET-1CEST,M3.5.0,M10.5.0/3"
+
+// need a WebServer for http access on port 80.
+WebServer server(80);
+
+// The text of builtin files are in this header file
+#include "builtinfiles.h"
+
+// enable the CUSTOM_ETAG_CALC to enable calculation of ETags by a custom function
+#define CUSTOM_ETAG_CALC
+
+// ===== Simple functions used to answer simple GET requests =====
+
+// This function is called when the WebServer was requested without giving a filename.
+// This will redirect to the file index.htm when it is existing otherwise to the built-in $upload.htm page
+void handleRedirect() {
+  TRACE("Redirect...\n");
+  String url = "/index.htm";
+
+  if (!LittleFS.exists(url)) { url = "/$upload.htm"; }
+
+  server.sendHeader("Location", url, true);
+  server.send(302);
+}  // handleRedirect()
+
+
+// This function is called when the WebServer was requested to list all existing files in the filesystem.
+// a JSON array with file information is returned.
+void handleListFiles() {
+  File dir = LittleFS.open("/", "r");
+  String result;
+
+  result += "[\n";
+  while (File entry = dir.openNextFile()) {
+    if (result.length() > 4) { result += ",\n"; }
+    result += "  {";
+    result += "\"type\": \"file\", ";
+    result += "\"name\": \"" + String(entry.name()) + "\", ";
+    result += "\"size\": " + String(entry.size()) + ", ";
+    result += "\"time\": " + String(entry.getLastWrite());
+    result += "}";
+  }  // while
+
+  result += "\n]";
+  server.sendHeader("Cache-Control", "no-cache");
+  server.send(200, "text/javascript; charset=utf-8", result);
+}  // handleListFiles()
+
+
+// This function is called when the sysInfo service was requested.
+void handleSysInfo() {
+  String result;
+
+  result += "{\n";
+  result += "  \"Chip Model\": " + String(ESP.getChipModel()) + ",\n";
+  result += "  \"Chip Cores\": " + String(ESP.getChipCores()) + ",\n";
+  result += "  \"Chip Revision\": " + String(ESP.getChipRevision()) + ",\n";
+  result += "  \"flashSize\": " + String(ESP.getFlashChipSize()) + ",\n";
+  result += "  \"freeHeap\": " + String(ESP.getFreeHeap()) + ",\n";
+  result += "  \"fsTotalBytes\": " + String(LittleFS.totalBytes()) + ",\n";
+  result += "  \"fsUsedBytes\": " + String(LittleFS.usedBytes()) + ",\n";
+  result += "}";
+
+  server.sendHeader("Cache-Control", "no-cache");
+  server.send(200, "text/javascript; charset=utf-8", result);
+}  // handleSysInfo()
+
+
+// ===== Request Handler class used to answer more complex requests =====
+
+// The FileServerHandler is registered to the web server to support DELETE and UPLOAD of files into the filesystem.
+class FileServerHandler : public RequestHandler {
+public:
+  // @brief Construct a new File Server Handler object
+  // @param fs The file system to be used.
+  // @param path Path to the root folder in the file system that is used for serving static data down and upload.
+  // @param cache_header Cache Header to be used in replies.
+  FileServerHandler() {
+    TRACE("FileServerHandler is registered\n");
+  }
+
+
+  // @brief check incoming request. Can handle POST for uploads and DELETE.
+  // @param requestMethod method of the http request line.
+  // @param requestUri request ressource from the http request line.
+  // @return true when method can be handled.
+  bool canHandle(HTTPMethod requestMethod, String UNUSED uri) override {
+    return ((requestMethod == HTTP_POST) || (requestMethod == HTTP_DELETE));
+  }  // canHandle()
+
+
+  bool canUpload(String uri) override {
+    // only allow upload on root fs level.
+    return (uri == "/");
+  }  // canUpload()
+
+
+  bool handle(WebServer &server, HTTPMethod requestMethod, String requestUri) override {
+    // ensure that filename starts with '/'
+    String fName = requestUri;
+    if (!fName.startsWith("/")) { fName = "/" + fName; }
+
+    TRACE("handle %s\n", fName.c_str());
+
+    if (requestMethod == HTTP_POST) {
+      // all done in upload. no other forms.
+
+    } else if (requestMethod == HTTP_DELETE) {
+      if (LittleFS.exists(fName)) {
+        TRACE("DELETE %s\n", fName.c_str());
+        LittleFS.remove(fName);
+      }
+    }  // if
+
+    server.send(200);  // all done.
+    return (true);
+  }  // handle()
+
+
+  // uploading process
+  void
+  upload(WebServer UNUSED &server, String UNUSED _requestUri, HTTPUpload &upload) override {
+    // ensure that filename starts with '/'
+    static size_t uploadSize;
+
+    if (upload.status == UPLOAD_FILE_START) {
+      String fName = upload.filename;
+
+      // Open the file for writing
+      if (!fName.startsWith("/")) { fName = "/" + fName; }
+      TRACE("start uploading file %s...\n", fName.c_str());
+
+      if (LittleFS.exists(fName)) {
+        LittleFS.remove(fName);
+      }  // if
+      _fsUploadFile = LittleFS.open(fName, "w");
+      uploadSize = 0;
+
+    } else if (upload.status == UPLOAD_FILE_WRITE) {
+      // Write received bytes
+      if (_fsUploadFile) {
+        size_t written = _fsUploadFile.write(upload.buf, upload.currentSize);
+        if (written < upload.currentSize) {
+          // upload failed
+          TRACE("  write error!\n");
+          _fsUploadFile.close();
+
+          // delete file to free up space in filesystem
+          String fName = upload.filename;
+          if (!fName.startsWith("/")) { fName = "/" + fName; }
+          LittleFS.remove(fName);
+        }
+        uploadSize += upload.currentSize;
+        // TRACE("free:: %d of %d\n", LittleFS.usedBytes(), LittleFS.totalBytes());
+        // TRACE("written:: %d of %d\n", written, upload.currentSize);
+        // TRACE("totalSize: %d\n", upload.currentSize + upload.totalSize);
+      }  // if
+
+    } else if (upload.status == UPLOAD_FILE_END) {
+        TRACE("finished.\n");
+      // Close the file
+      if (_fsUploadFile) {
+        _fsUploadFile.close();
+        TRACE(" %d bytes uploaded.\n", upload.totalSize);
+      }
+    }  // if
+
+  }  // upload()
+
+
+protected:
+  File _fsUploadFile;
+};
+
+
+// Setup everything to make the webserver work.
+void setup(void) {
+  delay(3000);  // wait for serial monitor to start completely.
+
+  // Use Serial port for some trace information from the example
+  Serial.begin(115200);
+  Serial.setDebugOutput(false);
+
+  TRACE("Starting WebServer example...\n");
+
+  TRACE("Mounting the filesystem...\n");
+  if (!LittleFS.begin()) {
+    TRACE("could not mount the filesystem...\n");
+    delay(2000);
+    TRACE("formatting...\n");
+    LittleFS.format();
+    delay(2000);
+    TRACE("restart.\n");
+    delay(2000);
+    ESP.restart();
+  }
+
+  // allow to address the device by the given name e.g. http://webserver
+  WiFi.setHostname(HOSTNAME);
+
+  // start WiFI
+  WiFi.mode(WIFI_STA);
+  if (strlen(ssid) == 0) {
+    WiFi.begin();
+  } else {
+    WiFi.begin(ssid, passPhrase);
+  }
+
+  TRACE("Connect to WiFi...\n");
+  while (WiFi.status() != WL_CONNECTED) {
+    delay(500);
+    TRACE(".");
+  }
+  TRACE("connected.\n");
+
+  // Ask for the current time using NTP request builtin into ESP firmware.
+  TRACE("Setup ntp...\n");
+  configTzTime(TIMEZONE, "pool.ntp.org");
+
+  TRACE("Register redirect...\n");
+
+  // register a redirect handler when only domain name is given.
+  server.on("/", HTTP_GET, handleRedirect);
+
+  TRACE("Register service handlers...\n");
+
+  // serve a built-in htm page
+  server.on("/$upload.htm", []() {
+    server.send(200, "text/html", FPSTR(uploadContent));
+  });
+
+  // register some REST services
+  server.on("/api/list", HTTP_GET, handleListFiles);
+  server.on("/api/sysinfo", HTTP_GET, handleSysInfo);
+
+  TRACE("Register file system handlers...\n");
+
+  // UPLOAD and DELETE of files in the file system using a request handler.
+  server.addHandler(new FileServerHandler());
+
+  // // enable CORS header in webserver results
+  server.enableCORS(true);
+
+  // enable ETAG header in webserver results (used by serveStatic handler)
+#if defined(CUSTOM_ETAG_CALC)
+  // This is a fast custom eTag generator. It returns a value based on the time the file was updated like
+  // ETag: 63bbceb5
+  server.enableETag(true, [](FS &fs, const String &path) -> String {
+    File f = fs.open(path, "r");
+    String eTag = String(f.getLastWrite(), 16);  // use file modification timestamp to create ETag
+    f.close();
+    return (eTag);
+  });
+
+#else
+  // enable standard ETAG calculation using md5 checksum of file content.
+  server.enableETag(true);
+#endif
+
+  // serve all static files
+  server.serveStatic("/", LittleFS, "/");
+
+  TRACE("Register default (not found) answer...\n");
+
+  // handle cases when file is not found
+  server.onNotFound([]() {
+    // standard not found in browser.
+    server.send(404, "text/html", FPSTR(notFoundContent));
+  });
+
+  server.begin();
+
+  TRACE("open <http://%s> or <http://%s>\n",
+        WiFi.getHostname(),
+        WiFi.localIP().toString().c_str());
+}  // setup
+
+
+// run the server...
+void loop(void) {
+  server.handleClient();
+}  // loop()
+
+// end.

libraries/WebServer/examples/WebServer/builtinfiles.h

+/**
+ * @file builtinfiles.h
+ * @brief This file is part of the WebServer example for the ESP8266WebServer.
+ *  
+ * This file contains long, multiline text variables for  all builtin resources.
+ */
+
+// used for $upload.htm
+static const char uploadContent[] PROGMEM =
+R"==(
+<!doctype html>
+<html lang='en'>
+
+<head>
+  <meta charset="utf-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <title>Upload</title>
+</head>
+
+<body style="width:300px">
+  <h1>Upload</h1>
+  <div><a href="/">Home</a></div>
+  <hr>
+  <div id='zone' style='width:16em;height:12em;padding:10px;background-color:#ddd'>Drop files here...</div>
+
+  <script>
+    // allow drag&drop of file objects 
+    function dragHelper(e) {
+      e.stopPropagation();
+      e.preventDefault();
+    }
+
+    // allow drag&drop of file objects 
+    function dropped(e) {
+      dragHelper(e);
+      var fls = e.dataTransfer.files;
+      var formData = new FormData();
+      for (var i = 0; i < fls.length; i++) {
+        formData.append('file', fls[i], '/' + fls[i].name);
+      }
+      fetch('/', { method: 'POST', body: formData }).then(function () {
+        window.alert('done.');
+      });
+    }
+    var z = document.getElementById('zone');
+    z.addEventListener('dragenter', dragHelper, false);
+    z.addEventListener('dragover', dragHelper, false);
+    z.addEventListener('drop', dropped, false);
+  </script>
+</body>
+)==";
+
+// used for $upload.htm
+static const char notFoundContent[] PROGMEM = R"==(
+<html>
+<head>
+  <title>Ressource not found</title>
+</head>
+<body>
+  <p>The ressource was not found.</p>
+  <p><a href="/">Start again</a></p>
+</body>
+)==";

libraries/WebServer/examples/WebServer/data/files.htm

+<html>
+
+<head>
+  <title>Files</title>
+  <link Content-Type="text/css" href="/style.css" rel="stylesheet" />
+</head>
+
+<body>
+  <h1>Files on Server</h1>
+
+  <p>These files are available on the server to be opened or delete:</p>
+  <div id="list">
+  </div>
+
+  <script>
+    // load and display all files after page loading has finished
+    window.addEventListener("load", function () {
+      fetch('/api/list')
+        .then(function (result) { return result.json(); })
+        .then(function (e) {
+          var listObj = document.querySelector('#list');
+          e.forEach(function (f) {
+            var entry = document.createElement("div");
+            var nameObj = document.createElement("a");
+            nameObj.href = '/' + f.name;
+            nameObj.innerText = '/' + f.name;
+            entry.appendChild(nameObj)
+            
+            entry.appendChild(document.createTextNode(' (' + f.size + ') '));
+
+            var timeObj = document.createElement("span");
+            timeObj.innerText = (new Date(f.time*1000)).toLocaleString();
+            entry.appendChild(timeObj)
+            entry.appendChild(document.createTextNode(" "));
+
+            var delObj = document.createElement("span");
+            delObj.className = 'deleteFile';
+            delObj.innerText = ' [delete]';
+            entry.appendChild(delObj)
+
+            listObj.appendChild(entry)
+          });
+
+        })
+        .catch(function (err) {
+          window.alert(err);
+        });
+    });
+
+    window.addEventListener("click", function (evt) {
+      var t = evt.target;
+      if (t.className === 'deleteFile') {
+        var fname = t.parentElement.innerText;
+        fname = fname.split(' ')[0];
+        if (window.confirm("Delete " + fname + " ?")) {
+          fetch(fname, { method: 'DELETE' });
+          document.location.reload(false);
+        }
+      };
+    });
+
+  </script>
+</body>
+
+</html>

libraries/WebServer/examples/WebServer/data/index.htm

+<html>
+
+<head>
+  <title>HomePage</title>
+  <link Content-Type="text/css" href="/style.css" rel="stylesheet" />
+</head>
+
+<body>
+  <h1>Homepage of the WebServer Example</h1>
+
+  <p>The following pages are available:</p>
+  <ul>
+    <li><a href="/index.htm">/index.htm</a> - This page</li>
+    <li><a href="/files.htm">/files.htm</a> - Manage files on the server</li>
+    <li><a href="/$upload.htm">/$upload.htm</a> - Built-in upload utility</a></li>
+    <li><a href="/none.htm">/none.htm</a> - See the default response when files are not found.</a></li>
+  </ul>
+  
+  <p>The following REST services are available:</p>
+  <ul>
+    <li><a href="/api/sysinfo">/api/sysinfo</a> - Some system level information</a></li>
+    <li><a href="/api/list">/api/list</a> - Array of all files</a></li>
+  </ul>
+</body>
+</html>

libraries/WebServer/examples/WebServer/data/style.css

+html, body {
+  color: #111111; font-family: Arial, ui-sans-serif, sans-serif; font-size: 1em; background-color: #f0f0f0;
+}
+
+#list > div {
+  margin: 0 0 0.5rem 0;
+}
+
+a { color: inherit; cursor: pointer; }
+

libraries/WebServer/examples/WebServer/secrets.h

+// Secrets for your local home network
+
+// This is a "hard way" to configure your local WiFi network name and passphrase
+// into the source code and the uploaded sketch.
+// 
+// Using the WiFi Manager is preferred and avoids reprogramming when your network changes.
+// See https://homeding.github.io/#page=/wifimanager.md
+
+// ssid and passPhrase can be used when compiling for a specific environment as a 2. option.
+
+// add you wifi network name and PassPhrase or use WiFi Manager
+const char *ssid = "";
+const char *passPhrase = "";

libraries/WebServer/src/WebServer.cpp

@@ -38,6 +38,7 @@ static const char qop_auth[] PROGMEM = "qop=auth";
 static const char qop_auth_quoted[] PROGMEM = "qop=\"auth\"";
 static const char WWW_Authenticate[] = "WWW-Authenticate";
 static const char Content_Length[] = "Content-Length";
+static const char ETAG_HEADER[] = "If-None-Match";
 
 
 WebServer::WebServer(IPAddress addr, int port)
@@ -381,6 +382,11 @@ void WebServer::enableCrossOrigin(boolean value) {
   enableCORS(value);
 }
 
+void WebServer::enableETag(bool enable, ETagFunction fn) {
+  _eTagEnabled = enable;
+  _eTagFunction = fn;
+}
+
 void WebServer::_prepareHeader(String& response, int code, const char* content_type, size_t contentLength) {
     response = String(F("HTTP/1.")) + String(_currentVersion) + ' ';
     response += String(code);
@@ -585,13 +591,14 @@ String WebServer::header(String name) {
 }
 
 void WebServer::collectHeaders(const char* headerKeys[], const size_t headerKeysCount) {
-  _headerKeysCount = headerKeysCount + 1;
+  _headerKeysCount = headerKeysCount + 2;
   if (_currentHeaders)
      delete[]_currentHeaders;
   _currentHeaders = new RequestArgument[_headerKeysCount];
   _currentHeaders[0].key = FPSTR(AUTHORIZATION_HEADER);
-  for (int i = 1; i < _headerKeysCount; i++){
-    _currentHeaders[i].key = headerKeys[i-1];
+  _currentHeaders[1].key = FPSTR(ETAG_HEADER);
+  for (int i = 2; i < _headerKeysCount; i++){
+    _currentHeaders[i].key = headerKeys[i-2];
   }
 }
 

libraries/WebServer/src/WebServer.h

@@ -27,6 +27,7 @@
 #include <functional>
 #include <memory>
 #include <WiFi.h>
+#include <FS.h>
 #include "HTTP_Method.h"
 #include "Uri.h"
 
@@ -130,6 +131,8 @@ public:
   void enableDelay(boolean value);
   void enableCORS(boolean value = true);
   void enableCrossOrigin(boolean value = true);
+  typedef std::function<String(FS &fs, const String &fName)> ETagFunction;
+  void enableETag(bool enable, ETagFunction fn = nullptr);
 
   void setContentLength(const size_t contentLength);
   void sendHeader(const String& name, const String& value, bool first = false);
@@ -146,6 +149,9 @@ public:
     return _currentClient.write(file);
   }
 
+  bool             _eTagEnabled = false;
+  ETagFunction     _eTagFunction = nullptr;
+
 protected:
   virtual size_t _currentClientWrite(const char* b, size_t l) { return _currentClient.write( b, l ); }
   virtual size_t _currentClientWrite_P(PGM_P b, size_t l) { return _currentClient.write_P( b, l ); }

libraries/WebServer/src/detail/RequestHandlersImpl.h

@@ -5,6 +5,8 @@
 #include "mimetable.h"
 #include "WString.h"
 #include "Uri.h"
+#include <MD5Builder.h>
+#include <base64.h>
 
 using namespace mime;
 
@@ -91,6 +93,7 @@ public:
         log_v("StaticRequestHandler::handle: request=%s _uri=%s\r\n", requestUri.c_str(), _uri.c_str());
 
         String path(_path);
+        String eTagCode;
 
         if (!_isFile) {
             // Base URI doesn't point to a file.
@@ -117,9 +120,26 @@ public:
         if (!f || !f.available())
             return false;
 
+        if (server._eTagEnabled) {
+            if (server._eTagFunction) {
+                eTagCode = (server._eTagFunction)(_fs, path);
+            } else {
+                eTagCode = calcETag(_fs, path);
+            }
+
+            if (server.header("If-None-Match") == eTagCode) {
+                server.send(304);
+                return true;
+            }
+        }
+
         if (_cache_header.length() != 0)
             server.sendHeader("Cache-Control", _cache_header);
 
+        if ((server._eTagEnabled) && (eTagCode.length() > 0)) {
+            server.sendHeader("ETag", eTagCode);
+        }
+
         server.streamFile(f, contentType);
         return true;
     }
@@ -139,6 +159,26 @@ public:
         return String(buff);
     }
 
+    // calculate an ETag for a file in filesystem based on md5 checksum
+    // that can be used in the http headers - include quotes.
+    static String calcETag(FS &fs, const String &path) {
+        String result;
+
+        // calculate eTag using md5 checksum
+        uint8_t md5_buf[16];
+        File f = fs.open(path, "r");
+        MD5Builder calcMD5;
+        calcMD5.begin();
+        calcMD5.addStream(f, f.size());
+        calcMD5.calculate();
+        calcMD5.getBytes(md5_buf);
+        f.close();
+        // create a minimal-length eTag using base64 byte[]->text encoding.
+        result = "\"" + base64::encode(md5_buf, 16) + "\"";
+        return(result);
+    } // calcETag
+
+
 protected:
     FS _fs;
     String _uri;