Security

• Prevent API URLs and contact URLs from linking to unknown domains in API key signup e-mails: Someone could trigger an API key signup e-mail to a user with links to unexpected locations for the example API URL or "contact us" link. Thanks to @nuke11 for the bug bounty report. (api.data.gov#460)
• XSS issue in flash error messages from external login providers: Error messages from external login providers (eg, Google) could contain a cross-site scripting (XSS) vulnerability. (469572c)
• Prevent admin groups from having analytics permissions: If an admin account belonged to only admin groups that didn't have any analytics permissions, then it was possible they admin could inadvertently view all analytics data. (a4569a6)

Added

• Added packages for Ubuntu 18.04 and Debian 9: Pre-packaged binaries are now available for the latest Debian and Ubuntu LTS releases. (#432, #444)
• Elasticsearch V5, V6, and V7 compatibility: If using an external Elasticsearch database, API Umbrella now support Elasticsearch versions 5, 6, and 7. The elasticsearch.api_version must be adjusted accordingly. (#393)
• Elasticsearch SSL support: You can now point to an Elasticsearch URL over HTTPS. (a201220, a5a403f, d89960f)
• AWS Elasticsearch signing for IAM access control: There is an extra proxy layer to support using AWS Elasticsearch when using IAM for access control. (9ddce5e)
• Option to log all output to stdout/stderr: The log.destination: console option can be used to log all output to stdout/stderr instead of log files. This makes API Umbrella easier to run in containerized environments. (#376)
• Options to parse client IPs from different HTTP headers: If other proxies are present in front of API Umbrella, additional options have been added to parse the original client's IP address from a different HTTP header. (api.data.gov#429, #431)
• Option to perform HTTPS redirects on specific API URLs: The router.api_backend_required_https_regex_default configuration option can be used to force certain API URLs to redirect to HTTPS based on a regex. (api.data.gov#457)
• Configurable API user validation regexes: Regexes to validate e-mail addresses, first names, and last names for API key signups are now configurable (web.api_user.email_regex, web.api_user.first_name_exclude_regex, web.api_user.last_name_exclude_regex). (15f14f3, 1566eef)
• Configurable nginx log levels: Allow the log level of the nginx processes to be configurable (nginx.error_log_level). (2b0c8ac)
• Configurable log levels for nginx rate limiting: Allow the log level used for nginx rate limit messages to be configurable (router.global_rate_limits.ip_connections_log_level and router.global_rate_limits.ip_rate_log_level, a804e0c)
• Docker development environment: Add a Docker-based development environment for easier development setup.
• Experimental support for integrating automatic SSL certificate registration: Integrate lua-resty-auto-ssl for automatically handling SSL certificates. (2f6c5b5)

Changed

• Route API backend requests directly from Traffic Server: Routing to API backends has been simplified so it occurs directly from Traffic Server, instead of routing back through an extra nginx hop. This should improve efficiency, simplifies routing, and eliminates DNS-related code. (#410)
• Admin UI Upgrades: Upgrade the admin UI project from Ember 2.8 to Ember 3.9 and Bootstrap 3 to Bootstrap 4. This switch also moves all dependencies into NPM instead of Bower, and better uses ES6 syntax throughout the admin UI code. Integration tests have also been switched from PhantomJS to Selenium tests using headless Chrome. (#429, api.data.gov#434)
• Upgrade to GeoIP2 database: The legacy GeoIP data previously being used has been discontinued, so GeoIP2 is now being used for geo-locating IP addresses. (8f17dae, #440)
• Redirect all website content to HTTPS by default: All website requests now redirect to HTTPS by default. (b3a8abc, #407, api.data.gov#430)
• Improve HTTPS requirements for API requests to error earlier: When making an insecure API request, return an error about HTTPS being required before the API key requirement error. (api.data.gov#454)
• Improve filesystem permissions and use more restrictive umask: Ensure that the files generated by API Umbrella are only readable by the needed users, and not readable by other users that may have access to the server. (2e595ce)
• Increase size of allowed HTTP response header lengths: If an API backend returned very long HTTP headers, it could generate 502 errors. This increases the default size of allowed HTTP headers, and also makes this length configurable. (api.data.gov#461, #398)
• Improve the build process for better caching: The build process has been revamped to allow for better caching of the dependencies. (#409) https://github.com/NREL/api-umbrella/pull/414
• Output Traffic Server logs as text logs: Output Traffic Server's access log as a text log file, instead of a binary log. (bd7f9fa)
• Preload Lua modules in nginx to improve memory usage: Lua modules are now preloaded in the nginx master process to improve memory usage by the nginx workers. (031620a)
• Add more validations on API user names: Add additional validations to ensure a valid first and last name have been entered to help prevent signup spam.
• Improve Cache-Control responses for admin content: Use stricter Cache-Control settings for admin responses that should not be cached, and improve caching of asset files. (api.data.gov#425)
• Disable animations of admin analytics charts: Disable the animations of the charts in the admin analytics to improve responsiveness. (92c9351)
• Switch to Lua code for generating request IDs: Switch from ngx_txid to lua-resty-txid for generating the request IDs to reduce need for custom nginx modules. (9d2ebd4)
• Require multi-factor authentication for MAX.gov admin logins: If using MAX.gov for admin logins, multi-factor authentication is required by default. (api.data.gov#435)
• Upgrade bundled software dependencies:
  • Elasticsearch 2.4.5 -> 2.4.6
  • MongoDB 3.2.15 -> 3.2.22
  • OpenResty 1.11.2.4 -> 1.13.6.2
  • OpenSSL 1.0.2l -> 1.0.2r
  • Rails 4.2.9 -> 4.2.11.1
  • Rsyslog 8.27.0 -> 8.1904.0
  • Ruby 2.3.4 -> 2.4.6
  • Traffic Server 5.3.2 -> 8.0.3

Removed

• Removed Ubuntu 12.04, Ubuntu 14.014 and Debian 7 packages: Removed packages for unsupported distributions.
• Removed references to request_ip_location: Removed defunct references to the analytics request_ip_location field that was removed in v0.14.0. (c783e1c)
• Removed experimental analytics: Removed code related to experimental analytics backend. (77d50d0)
• Removed Vagrant development environment: Removed the Vagrant-based development environment in favor of the Docker-based development environment.