Merge branch 'release/1.1.8'

* ImageMedium->derivatives now works with image filters

Previously, using ImageMedium->derivatives would not work well in
combination with image filters or the other method of generating
srcset variants of images (by appending eg. "@2x" to their
filenames). This commit hopefully fixes that.

* Modified initialization of image alternatives

The biggest alternative will now become the base medium, and
alternatives will be filled out as necessary in a descending
manner.

* Fully reset image when derivatives method is called

Otherwise we get some funky results, with the possibility of two
different images being rendered between the full-width srcset
version and the original src version.

* Account for risk of original file not existing when generating image derivatives

* Fixed an issue where too many alternatives would be generated

When using naming conventions to generate image alternatives, this
patch would previously generate a “@1x” alternative if one didn’t
exist. That’s no longer the case

* Add an "@1x" alternative when an image lacks a base medium

When an image only has an alternative medium - ie. the only
version of the image ends in eg. "@3x", then we construct the
missing alternatives automatically. Previously, we would only do
this down till "@2x", meaning that the image that would have been
the base medium, had the image been manually resized, wasn't
created. This has now been fixed.

* Always make smallest image alternative the base medium

When an image lacks a base medium on disk (eg. the only existing
image is an @2x version), then we make a scaled down version the
base medium, which ensures that the smaller version is served up
in the src attribute in the HTML.

Also, don't reset the image alternatives when calling
ImageMedium#derivatives, instead only generate the image
alternatives that are missing.

* Set better prettynames for image derivatives

* Changed image derivatives prettynames to be width based

Instead of example2x.jpeg, we now have eg. example1280w.jpeg

.gitignore

@@ -1,6 +1,7 @@
 # Composer
 .composer
-/vendor
+vendor/*
+!*/vendor/*
 
 # Sass
 .sass-cache

.htaccess

@@ -13,6 +13,16 @@ RewriteEngine On
 
 ## End - RewriteBase
 
+## Begin - X-Forwarded-Proto
+# In some hosted or load balanced environments, SSL negotiation happens upstream.
+# In order for Grav to recognize the connection as secure, you need to uncomment
+# the following lines.
+#
+# RewriteCond %{HTTP:X-Forwarded-Proto} https
+# RewriteRule .* - [E=HTTPS:on]
+#
+## End - X-Forwarded-Proto
+
 ## Begin - Exploits
 # If you experience problems on your site block out the operations listed below
 # This attempts to block the most common type of exploit `attempts` to Grav

.travis.yml

@@ -43,8 +43,6 @@ env:
     - secure: "einUtSEkUWy2IrqLXyVjwUU+mwaaoiOXRRVdLBpA3Zye6bZx8cm5h/5AplkPWhM/NmCJoW/MwNZHHkFhlr3mDRov5iOxVmTTYfnXB+I5lxYTSgduOLLErS7mU8hfADpVDU8bHNU44fNGD3UEiG1PD4qQBX4DMlqIFmR20mjs81k="
     # GH_API_USER [for curl]
     - secure: "AQGcX1B2NrI8ajflY4AimZDNcK2kBA3F6mbtEFQ78NkDoWhMipsQHayWXiSTzRc0YJKvQl2Y16MTwQF4VHzjTAiiZFATgA8J88vQUjIPabi/kKjqSmcLFoaAOAxStQbW6e0z2GiQ6KBMcNF1y5iUuI63xVrBvtKrYX/w5y+ako8="
-    # Latest Release version
-    - TRAVIS_TAG=$(curl --fail -s https://api.github.com/repos/getgrav/grav/releases/latest | grep tag_name | head -n 1 | cut -d '"' -f 4)
 
 before_install:
   - export TZ=Pacific/Honolulu
@@ -56,6 +54,7 @@ before_install:
         composer install --dev --prefer-dist;
     fi
   - if [ $TRAVIS_BRANCH != 'develop' ] && [ $TRAVIS_PHP_VERSION == "5.6" ] && [ $TRAVIS_PULL_REQUEST == "false" ]; then
+        export TRAVIS_TAG=$(curl --fail --user "${GH_API_USER}" -s https://api.github.com/repos/getgrav/grav/releases/latest | grep tag_name | head -n 1 | cut -d '"' -f 4);
         go get github.com/aktau/github-release;
         git clone --quiet --depth=50 --branch=master https://${BB_TOKEN}bitbucket.org/rockettheme/grav-devtools.git $RT_DEVTOOLS  &>/dev/null;
         if [ ! -z "$TRAVIS_TAG" ]; then
@@ -69,7 +68,7 @@ script:
   - if [ $TRAVIS_BRANCH == 'develop' ] || [ $TRAVIS_PULL_REQUEST != 'false' ]; then
         vendor/bin/codecept run;
     fi
-  - echo $TRAVIS_TAG
+  - echo "Latest Release Tag - ${TRAVIS_TAG}"
   - if [ ! -z "$TRAVIS_TAG" ] && [ $TRAVIS_BRANCH != 'develop' ] && [ $TRAVIS_PHP_VERSION == "5.6" ] && [ $TRAVIS_PULL_REQUEST == "false" ]; then
       FILES="$RT_DEVTOOLS/grav-dist/*.zip";
       for file in ${FILES[@]}; do

CHANGELOG.md

@@ -1,3 +1,99 @@
+# v1.1.8
+## 10/22/2016
+
+1. [](#bugfix)
+    * Fixed warning with unset `ssl` option when using GPM [#1132](https://github.com/getgrav/grav/issues/1132)
+
+# v1.1.7
+## 10/22/2016
+
+1. [](#improved)
+    * Improved the capabilities of Image derivatives [#1107](https://github.com/getgrav/grav/pull/1107)
+1. [](#bugfix)
+    * Only pass verify_peer settings to cURL and fopen if the setting is disabled [#1120](https://github.com/getgrav/grav/issues/1120)
+
+# v1.1.6
+## 10/19/2016
+
+1. [](#new)
+    * Added ability for Page to override the output format (`html`, `xml`, etc..) [#1067](https://github.com/getgrav/grav/issues/1067)
+    * Added `Utils::getExtensionByMime()` and cleaned up `Utils::getMimeByExtension` + tests
+    * Added a `cache.check.method: 'hash'` option in `system.yaml` that checks all files + dates inclusively
+    * Include jQuery 3.x in the Grav assets
+    * Added the option to automatically fix orientation on images based on their Exif data, by enabling `system.images.auto_fix_orientation`.
+1. [](#improved)
+    * Add `batch()` function to Page Collection class
+    * Added new `cache.redis.socket` setting that allow to pass a UNIX socket as redis server
+    * It is now possible to opt-out of the SSL verification via the new `system.gpm.verify_peer` setting. This is sometimes necessary when receiving a "GPM Unable to Connect" error. More details in ([#1053](https://github.com/getgrav/grav/issues/1053))
+    * It is now possible to force the use of either `curl` or `fopen` as `Response` connection method, via the new `system.gpm.method` setting. By default this is set to 'auto' and gives priority to 'fopen' first, curl otherwise.
+    * InstallCommand can now handle Licenses
+    * Uses more helpful `1x`, `2x`, `3x`, etc names in the Retina derivatives cache files.
+    * Added new method `Plugins::isPluginActiveAdmin()` to check if plugin route is active in Admin plugin
+    * Added new `Cache::setEnabled` and `Cache::getEnabled` to enable outside control of cache
+    * Updated vendor libs including Twig `1.25.0`
+    * Avoid git ignoring any vendor folder in a Grav site subfolder (but still ignore the main `vendor/` folder)
+    * Added an option to get just a route back from `Uri::convertUrl()` function
+    * Added option to control split session [#1096](https://github.com/getgrav/grav/pull/1096)
+    * Added new `verbosity` levels to `system.error.display` to allow for system error messages [#1091](https://github.com/getgrav/grav/pull/1091)
+    * Improved the API for Grav plugins to access the Parsedown parser directly [#1062](https://github.com/getgrav/grav/pull/1062)
+1. [](#bugfix)
+    * Fixed missing `progress` method in the DirectInstall Command
+    * `Response` class now handles better unsuccessful requests such as 404 and 401
+    * Fixed saving of `external` page types [admin #789](https://github.com/getgrav/grav-plugin-admin/issues/789)
+    * Fixed issue deleting parent folder of folder with `param_sep` in the folder name [admin #796](https://github.com/getgrav/grav-plugin-admin/issues/796)
+    * Fixed an issue with streams in `bin/plugin`
+    * Fixed `jpeg` file format support in Media
+
+# v1.1.5
+## 09/09/2016
+
+1. [](#new)
+    * Added new `bin/gpm direct-install` command to install local and remote zip archives
+1. [](#improved)
+    * Refactored `onPageNotFound` event to fire after `onPageInitialized`
+    * Follow symlinks in `Folder::all()`
+    * Twig variable `base_url` now supports multi-site by path feature
+    * Improved `bin/plugin` to list plugins with commands faster by limiting the depth of recursion
+1. [](#bugfix)
+    * Quietly skip missing streams in `Cache::clearCache()`
+    * Fix issue in calling page.summary when no content is present in a page
+    * Fix for HUGE session timeouts [#1050](https://github.com/getgrav/grav/issues/1050)
+
+# v1.1.4
+## 09/07/2016
+
+1. [](#new)
+    * Added new `tmp` folder at root. Accessible via stream `tmp://`. Can be cleared with `bin/grav clear --tmp-only` as well as `--all`.
+    * Added support for RTL in `LanguageCodes` so you can determine if a language is RTL or not
+    * Ability to set `custom_base_url` in system configuration
+    * Added `override` and `force` options for Streams setup
+1. [](#improved)
+    * Important vendor updates to provide PHP 7.1 beta support!
+    * Added a `Util::arrayFlatten()` static function
+    * Added support for 'external_url' page header to enable easier external URL based menu items
+    * Improved the UI for CLI GPM Index view to use a table
+    * Added `@page.modular` Collection type [#988](https://github.com/getgrav/grav/issues/988)
+    * Added support for `self@`, `page@`, `taxonomy@`, `root@` Collection syntax for cleaner YAML compatibility
+    * Improved GPM commands to allow for `-y` to automate **yes** responses and `-o` for **update** and **selfupgrade** to overwrite installations [#985](https://github.com/getgrav/grav/issues/985)
+    * Added randomization to `safe_email` Twig filter for greater security [#998](https://github.com/getgrav/grav/issues/998)
+    * Allow `Utils::setDotNotation` to merge data, rather than just set
+    * Moved default `Image::filter()` to the `save` action to ensure they are applied last [#984](https://github.com/getgrav/grav/issues/984)
+    * Improved the `Truncator` code to be more reliable [#1019](https://github.com/getgrav/grav/issues/1019)
+    * Moved media blueprints out of core (now in Admin plugin)
+1. [](#bugfix)
+    * Removed 307 redirect code option as it is not well supported [#743](https://github.com/getgrav/grav-plugin-admin/issues/743)
+    * Fixed issue with folders with name `*.md` are not confused with pages [#995](https://github.com/getgrav/grav/issues/995)
+    * Fixed an issue when filtering collections causing null key
+    * Fix for invalid HTML when rendering GIF and Vector media [#1001](https://github.com/getgrav/grav/issues/1001)
+    * Use pages.markdown.extra in the user's system.yaml [#1007](https://github.com/getgrav/grav/issues/1007)
+    * Fix for `Memcached` connection [#1020](https://github.com/getgrav/grav/issues/1020)
+
+# v1.1.3
+## 08/14/2016
+
+1. [](#bugfix)
+    * Fix for lightbox media function throwing error [#981](https://github.com/getgrav/grav/issues/981)
+
 # v1.1.2
 ## 08/10/2016
 

bin/gpm

@@ -7,6 +7,7 @@ if (!file_exists(__DIR__ . '/../vendor')){
 }
 
 use Grav\Common\Composer;
+use Grav\Common\Config\Setup;
 
 if (!file_exists(__DIR__ . '/../vendor')){
     // Before we can even start, we need to run composer first
@@ -37,9 +38,24 @@ if (!function_exists('curl_version')) {
     exit('FATAL: GPM requires PHP Curl module to be installed');
 }
 
+$climate = new League\CLImate\CLImate;
+$climate->arguments->add([
+    'environment' => [
+        'prefix'        => 'e',
+        'longPrefix'    => 'env',
+        'description'   => 'Configuration Environment',
+        'defaultValue'  => 'localhost'
+    ]
+]);
+$climate->arguments->parse();
+$environment = $climate->arguments->get('environment');
+
+// Set up environment based on params.
+Setup::$environment = $environment;
+
 $grav = Grav::instance(array('loader' => $autoload));
-$grav['config']->init();
 $grav['uri']->init();
+$grav['config']->init();
 $grav['streams'];
 
 $app = new Application('Grav Package Manager', GRAV_VERSION);
@@ -51,7 +67,7 @@ $app->addCommands(array(
     new \Grav\Console\Gpm\UninstallCommand(),
     new \Grav\Console\Gpm\UpdateCommand(),
     new \Grav\Console\Gpm\SelfupgradeCommand(),
+    new \Grav\Console\Gpm\DirectInstallCommand(),
 ));
 
-$app->setDefaultCommand('index');
 $app->run();

bin/plugin

@@ -21,6 +21,7 @@ use Symfony\Component\Console\Input\ArgvInput;
 use Symfony\Component\Console\Output\ConsoleOutput;
 use Symfony\Component\Console\Formatter\OutputFormatterStyle;
 use Grav\Common\Grav;
+use Grav\Common\Config\Setup;
 use Grav\Common\Filesystem\Folder;
 
 $autoload = require_once(__DIR__ . '/../vendor/autoload.php');
@@ -37,12 +38,29 @@ if (!file_exists(ROOT_DIR . 'index.php')) {
     exit('FATAL: Must be run from ROOT directory of Grav!');
 }
 
+$climate = new League\CLImate\CLImate;
+$climate->arguments->add([
+    'environment' => [
+        'prefix'        => 'e',
+        'longPrefix'    => 'env',
+        'description'   => 'Configuration Environment',
+        'defaultValue'  => 'localhost'
+    ]
+]);
+$climate->arguments->parse();
+$environment = $climate->arguments->get('environment');
+
+// Set up environment based on params.
+Setup::$environment = $environment;
+
 $grav = Grav::instance(array('loader' => $autoload));
+$grav['uri']->init();
 $grav['config']->init();
 $grav['streams'];
 $grav['plugins']->init();
 $grav['themes']->init();
 
+
 $app     = new Application('Grav Plugins Commands', GRAV_VERSION);
 $pattern = '([A-Z]\w+Command\.php)';
 
@@ -70,7 +88,7 @@ if (!$name) {
     $output->writeln('');
     $output->writeln("<red>Example:</red>");
     $output->writeln("  {$bin} error log -l 1 --trace");
-    $list = Folder::all('plugins://', ['compare' => 'Pathname', 'pattern' => '/\/cli\/' . $pattern . '$/usm']);
+    $list = Folder::all('plugins://', ['compare' => 'Pathname', 'pattern' => '/\/cli\/' . $pattern . '$/usm', 'levels' => 2]);
 
     if (count($list)) {
         $available = [];
@@ -101,14 +119,15 @@ if ($plugin === null) {
 $path = 'plugins://' . $name . '/cli';
 
 try {
-    $commands = Folder::all($path, ['compare' => 'Filename', 'pattern' => '/' . $pattern . '$/usm']);
+    $commands = Folder::all($path, ['compare' => 'Filename', 'pattern' => '/' . $pattern . '$/usm', 'levels' => 1]);
 } catch (\RuntimeException $e) {
     $output->writeln("<red>No Console Commands for <white>'{$name}'</white> where found in <white>'{$path}'</white></red>");
     exit;
 }
 
 foreach ($commands as $command_path) {
-    require_once "plugins://{$name}/cli/{$command_path}";
+    $full_path = $grav['locator']->findResource("plugins://{$name}/cli/{$command_path}");
+    require_once $full_path;
 
     $command_class = 'Grav\Plugin\Console\\' . preg_replace('/.php$/', '', $command_path);
     $command = new $command_class();

composer.json

@@ -19,15 +19,17 @@
         "filp/whoops": "~2.0",
         "matthiasmullie/minify": "^1.3",
         "monolog/monolog": "~1.0",
-        "gregwar/image": "~2.0",
+        "gregwar/image": "dev-master#72568cfbeb77515278f2ccb386fc344e874b7ae8",
         "donatj/phpuseragentparser": "~0.3",
         "pimple/pimple": "~3.0",
-        "rockettheme/toolbox": "dev-develop",
+        "rockettheme/toolbox": "~1.0",
         "maximebf/debugbar": "~1.10",
         "ext-mbstring": "*",
         "ext-openssl": "*",
         "ext-curl": "*",
-        "ext-zip": "*"
+        "ext-zip": "*",
+        "league/climate": "^3.2",
+        "antoligy/dom-string-iterators": "^1.0"
     },
     "require-dev": {
         "codeception/codeception": "^2.1",

system/blueprints/config/system.yaml

@@ -178,8 +178,9 @@ form:
                     help: PLUGIN_ADMIN.REDIRECT_DEFAULT_CODE_HELP
                     options:
                         301: 301 - Permanent
+                        302: 302 - Found
                         303: 303 - Other
-                        307: 307 - Temporary
+                        304: 304 - Not Modified
 
                 pages.redirect_trailing_slash:
                     type: toggle
@@ -514,6 +515,13 @@ form:
                     help: PLUGIN_ADMIN.MEMCACHED_PORT_HELP
                     placeholder: "11211"
 
+                cache.redis.socket:
+                    type: text
+                    size: medium
+                    label: PLUGIN_ADMIN.REDIS_SOCKET
+                    help: PLUGIN_ADMIN.REDIS_SOCKET_HELP
+                    placeholder: "/var/run/redis/redis.sock"
+
                 cache.redis.server:
                     type: text
                     size: medium
@@ -756,15 +764,16 @@ form:
 
             fields:
                 errors.display:
-                    type: toggle
+                    type: select
                     label: PLUGIN_ADMIN.DISPLAY_ERRORS
                     help: PLUGIN_ADMIN.DISPLAY_ERRORS_HELP
-                    highlight: 0
+                    size: medium
+                    highlight: 1
                     options:
-                        1: PLUGIN_ADMIN.YES
-                        0: PLUGIN_ADMIN.NO
-                    validate:
-                        type: bool
+                        -1: PLUGIN_ADMIN.ERROR_SYSTEM
+                        0: PLUGIN_ADMIN.ERROR_SIMPLE
+                        1: PLUGIN_ADMIN.ERROR_FULL_BACKTRACE
+
 
                 errors.log:
                     type: toggle
@@ -843,7 +852,6 @@ form:
                         '0755': '0755'
                         '0775': '0775'
 
-
                 images.debug:
                     type: toggle
                     label: PLUGIN_ADMIN.IMAGES_DEBUG
@@ -855,6 +863,17 @@ form:
                     validate:
                         type: bool
 
+                images.auto_fix_orientation:
+                    type: toggle
+                    label: PLUGIN_ADMIN.IMAGES_AUTO_FIX_ORIENTATION
+                    help: PLUGIN_ADMIN.IMAGES_AUTO_FIX_ORIENTATION_HELP
+                    highlight: 0
+                    options:
+                        1: PLUGIN_ADMIN.YES
+                        0: PLUGIN_ADMIN.NO
+                    validate:
+                        type: bool
+
                 media.upload_limit:
                     type: text
                     append: bytes
@@ -953,6 +972,24 @@ form:
                     validate:
                         type: bool
 
+                session.path:
+                    type: text
+                    size: small
+                    label: PLUGIN_ADMIN.SESSION_PATH
+                    help: PLUGIN_ADMIN.SESSION_PATH_HELP
+
+                session.split:
+                    type: toggle
+                    label: PLUGIN_ADMIN.SESSION_SPLIT
+                    help: PLUGIN_ADMIN.SESSION_SPLIT_HELP
+                    highlight: 1
+                    options:
+                        1: PLUGIN_ADMIN.YES
+                        0: PLUGIN_ADMIN.NO
+                    default: true
+                    validate:
+                        type: bool
+
         advanced:
             type: section
             title: PLUGIN_ADMIN.ADVANCED
@@ -975,6 +1012,27 @@ form:
                     label: PLUGIN_ADMIN.PROXY_URL
                     help: PLUGIN_ADMIN.PROXY_URL_HELP
 
+                gpm.method:
+                    type: toggle
+                    label: PLUGIN_ADMIN.GPM_METHOD
+                    highlight: auto
+                    help: PLUGIN_ADMIN.GPM_METHOD_HELP
+                    options:
+                        auto: PLUGIN_ADMIN.AUTO
+                        fopen: PLUGIN_ADMIN.FOPEN
+                        curl: PLUGIN_ADMIN.CURL
+
+                gpm.verify_peer:
+                    type: toggle
+                    label: PLUGIN_ADMIN.GPM_VERIFY_PEER
+                    highlight: 1
+                    help: PLUGIN_ADMIN.GPM_VERIFY_PEER_HELP
+                    options:
+                        1: PLUGIN_ADMIN.YES
+                        0: PLUGIN_ADMIN.NO
+                    validate:
+                        type: bool
+
                 reverse_proxy_setup:
                     type: toggle
                     label: PLUGIN_ADMIN.REVERSE_PROXY
@@ -1029,3 +1087,10 @@ form:
                         0: PLUGIN_ADMIN.NO
                     validate:
                         type: bool
+
+                custom_base_url:
+                    type: text
+                    size: medium
+                    placeholder: "e.g. http://localhost:8080"
+                    label: PLUGIN_ADMIN.CUSTOM_BASE_URL
+                    help: PLUGIN_ADMIN.CUSTOM_BASE_URL_HELP

system/blueprints/media/meta.yaml

@@ -1,7 +0,0 @@
-form:
-  validation: loose
-  fields:
-
-    alt_text:
-      type: string
-      label: Alt Text

system/blueprints/media/move.yaml

@@ -1,8 +0,0 @@
-form:
-  validation: loose
-  fields:
-    route:
-      type: select
-      label: PLUGIN_ADMIN.PAGE
-      classes: fancy
-      data-options@: '\Grav\Common\Page\Pages::parents'

system/blueprints/media/rename.yaml

@@ -1,8 +0,0 @@
-form:
-  validation: loose
-  fields:
-    new_file_name:
-      type: text
-      label: PLUGIN_ADMIN_PRO.NEW_FILE_NAME
-      validate:
-        required: true

system/blueprints/pages/external.yaml

@@ -0,0 +1,58 @@
+title: PLUGIN_ADMIN:EXTERNAL
+@extends:
+    type: default
+    context: blueprints://pages
+
+rules:
+  slug:
+    pattern: "[a-zа-я][a-zа-я0-9_\-]+"
+    min: 2
+    max: 80
+
+form:
+  validation: loose
+  fields:
+
+    tabs:
+      type: tabs
+      active: 1
+
+      fields:
+
+        content:
+          fields:
+
+            header.title:
+              type: text
+              autofocus: true
+              style: horizontal
+              label: PLUGIN_ADMIN.TITLE
+
+            content:
+              unset@: true
+
+            uploads:
+              unset@: true
+
+            header.external_url:
+                type: text
+                label: PLUGIN_ADMIN.EXTERNAL_URL
+                placeholder: https://getgrav.org
+                validate:
+                    required: true
+        options:
+          fields:
+
+            publishing:
+
+              fields:
+
+                header.date:
+                  unset@: true
+
+                header.metadata:
+                  unset@: true
+
+            taxonomies:
+              unset@: true
+

system/config/media.yaml

@@ -114,6 +114,9 @@ types:
   xlm:
     type: file
     mime: application/vnd.ms-excel
+  xlsm:
+    type: file
+    mime: application/vnd.ms-excel
   xld:
     type: file
     mime: application/vnd.ms-excel

system/config/system.yaml

@@ -4,7 +4,8 @@ default_locale:                             # Default locale (defaults to system
 param_sep: ':'                              # Parameter separator, use ';' for Apache on windows
 wrapped_site: false                         # For themes/plugins to know if Grav is wrapped by another platform
 reverse_proxy_setup: false                  # Running in a reverse proxy scenario with different webserver ports than proxy
-force_ssl: false                            # If enabled, Grav forces to be accessed via HTTPS
+force_ssl: false                            # If enabled, Grav forces to be accessed via HTTPS (NOTE: Not an ideal solution)
+custom_base_url: ''                         # Set the base_url manually
 
 languages:
   supported: []                             # List of languages supported. eg: [en, fr, de]
@@ -37,7 +38,7 @@ pages:
   twig_first: false                         # Process Twig before markdown when processing both on a page
   events:
     page: true                              # Enable page level events
-    twig: true                              # Enable twig level events
+    twig: true                              # Enable Twig level events
   markdown:
     extra: false                            # Enable support for Markdown Extra support (GFM by default)
     auto_line_breaks: false                 # Enable automatic line breaks
@@ -61,19 +62,21 @@ pages:
   url_taxonomy_filters: true                # Enable auto-magic URL-based taxonomy filters for page collections
   frontmatter:
     process_twig: false                     # Should the frontmatter be processed to replace Twig variables?
-    ignore_fields: ['form']                 # Fields that might contain Twig variables and should not be processed
+    ignore_fields: ['form','forms']         # Fields that might contain Twig variables and should not be processed
 
 cache:
   enabled: true                             # Set to true to enable caching
   check:
-    method: file                            # Method to check for updates in pages: file|folder|none
+    method: file                            # Method to check for updates in pages: file|folder|hash|none
   driver: auto                              # One of: auto|file|apc|xcache|memcache|wincache
   prefix: 'g'                               # Cache prefix string (prevents cache conflicts)
   lifetime: 604800                          # Lifetime of cached data in seconds (0 = infinite)
   gzip: false                               # GZip compress the page output
+  redis:
+    socket: false                           # Path to redis unix socket (e.g. /var/run/redis/redis.sock), false = use server and port to connect
 
 twig:
-  cache: true                               # Set to true to enable twig caching
+  cache: true                               # Set to true to enable Twig caching
   debug: true                               # Enable Twig debug
   auto_reload: true                         # Refresh cache on changes
   autoescape: false                         # Autoescape Twig vars
@@ -97,7 +100,7 @@ assets:                                     # Configuration for Assets Manager (
     jquery: system://assets/jquery/jquery-2.x.min.js
 
 errors:
-  display: false                            # Display full backtrace-style error page
+  display: 0                                # Display either (1) Full backtrace | (0) Simple Error | (-1) System Error
   log: true                                 # Log errors to /logs folder
 
 debugger:
@@ -110,6 +113,7 @@ images:
   cache_all: false                          # Cache all image by default
   cache_perms: '0755'                       # MUST BE IN QUOTES!! Default cache folder perms. Usually '0755' or '0775'
   debug: false                              # Show an overlay over images indicating the pixel depth of the image when working with retina for example
+  auto_fix_orientation: false               # Automatically fix the image orientation based on the Exif data
 
 media:
   enable_media_timestamp: false             # Enable media timetsamps
@@ -123,7 +127,11 @@ session:
   name: grav-site                           # Name prefix of the session cookie. Use alphanumeric, dashes or underscores only. Do not use dots in the session name
   secure: false                             # Set session secure. If true, indicates that communication for this cookie must be over an encrypted transmission. Enable this only on sites that run exclusively on HTTPS
   httponly: true                            # Set session HTTP only. If true, indicates that cookies should be used only over HTTP, and JavaScript modification is not allowed.
+  split: true                               # Sessions should be independent between site and plugins (such as admin)
+  path:
 
 gpm:
   releases: stable                           # Set to either 'stable' or 'testing'
   proxy_url:                                 # Configure a manual proxy URL for GPM (eg 127.0.0.1:3128)
+  method: 'auto'                             # Either 'curl', 'fopen' or 'auto'. 'auto' will try fopen first and if not available cURL
+  verify_peer: true                          # Sometimes on some systems (Windows most commonly) GPM is unable to connect because the SSL certificate cannot be verified. Disabling this setting might help.

system/defines.php

@@ -8,7 +8,7 @@
 
 // Some standard defines
 define('GRAV', true);
-define('GRAV_VERSION', '1.1.2');
+define('GRAV_VERSION', '1.1.8');
 define('GRAV_TESTING', false);
 define('DS', '/');
 define('GRAV_PHP_MIN', '5.5.9');

system/pages/notfound.md

@@ -0,0 +1,5 @@
+---
+title: Not Found
+routable: false
+notfound: true
+---

system/src/Grav/Common/Assets.php

@@ -207,7 +207,7 @@ class Assets
         $this->assets_url = $locator->findResource('asset://', false);
 
         $this->config($asset_config);
-        $this->base_url = $base_url . '/';
+        $this->base_url = ($config->get('system.absolute_urls') ? '' : '/') . ltrim(ltrim($base_url, '/') . '/', '/');
 
         // Register any preconfigured collections
         foreach ($config->get('system.assets.collections', []) as $name => $collection) {
@@ -1349,7 +1349,7 @@ class Assets
     public function getTimestamp($asset = null)
     {
         if (is_array($asset)) {
-            if ($asset['remote'] == false) {
+            if ($asset['remote'] === false) {
                 if (Utils::contains($asset['asset'], '?')) {
                     return str_replace('?', '&', $this->timestamp);
                 } else {

system/src/Grav/Common/Backup/ZipBackup.php

@@ -17,7 +17,8 @@ class ZipBackup
         'backup',
         'cache',
         'images',
-        'logs'
+        'logs',
+        'tmp'
     ];
 
     protected static $ignoreFolders = [

system/src/Grav/Common/Cache.php

@@ -65,7 +65,8 @@ class Cache extends Getters
     protected static $all_remove = [
         'cache://',
         'cache://images',
-        'asset://'
+        'asset://',
+        'tmp://'
     ];
 
     protected static $assets_remove = [
@@ -80,6 +81,10 @@ class Cache extends Getters
         'cache://'
     ];
 
+    protected static $tmp_remove = [
+        'tmp://'
+    ];
+
     /**
      * Constructor
      *
@@ -110,7 +115,9 @@ class Cache extends Getters
 
         $prefix = $this->config->get('system.cache.prefix');
 
-        $this->enabled = (bool)$this->config->get('system.cache.enabled');
+        if (is_null($this->enabled)) {
+            $this->enabled = (bool)$this->config->get('system.cache.enabled');
+        }
 
         // Cache key allows us to invalidate all cache on configuration changes.
         $this->key = ($prefix ? $prefix : 'g') . '-' . substr(md5($uri->rootUrl(true) . $this->config->key() . GRAV_VERSION),
@@ -122,10 +129,36 @@ class Cache extends Getters
 
         // Set the cache namespace to our unique key
         $this->driver->setNamespace($this->key);
+    }
 
-        // Dump Cache state
-        $grav['debugger']->addMessage('Cache: [' . ($this->enabled ? 'true' : 'false') . '] Setting: [' . $this->driver_setting . '] Driver: [' . $this->driver_name . ']');
+    /**
+     * Public accessor to set the enabled state of the cache
+     *
+     * @param $enabled
+     */
+    public function setEnabled($enabled)
+    {
+        $this->enabled = (bool) $enabled;
+    }
 
+    /**
+     * Returns the current enabled state
+     *
+     * @return bool
+     */
+    public function getEnabled()
+    {
+        return $this->enabled;
+    }
+
+    /**
+     * Get cache state
+     *
+     * @return string
+     */
+    public function getCacheStatus()
+    {
+        return 'Cache: [' . ($this->enabled ? 'true' : 'false') . '] Setting: [' . $this->driver_setting . '] Driver: [' . $this->driver_name . ']';
     }
 
     /**
@@ -183,7 +216,7 @@ class Cache extends Getters
 
             case 'memcached':
                 $memcached = new \Memcached();
-                $memcached->connect($this->config->get('system.cache.memcached.server', 'localhost'),
+                $memcached->addServer($this->config->get('system.cache.memcached.server', 'localhost'),
                     $this->config->get('system.cache.memcached.port', 11211));
                 $driver = new DoctrineCache\MemcachedCache();
                 $driver->setMemcached($memcached);
@@ -191,8 +224,15 @@ class Cache extends Getters
 
             case 'redis':
                 $redis = new \Redis();
-                $redis->connect($this->config->get('system.cache.redis.server', 'localhost'),
+                $socket = $this->config->get('system.cache.redis.socket', false);
+
+                if ($socket) {
+                    $redis->connect($socket);
+                } else {
+                    $redis->connect($this->config->get('system.cache.redis.server', 'localhost'),
                     $this->config->get('system.cache.redis.port', 6379));
+                }
+
                 $driver = new DoctrineCache\RedisCache();
                 $driver->setRedis($redis);
                 break;
@@ -309,6 +349,9 @@ class Cache extends Getters
             case 'cache-only':
                 $remove_paths = self::$cache_remove;
                 break;
+            case 'tmp-only':
+                $remove_paths = self::$tmp_remove;
+                break;
             default:
                 $remove_paths = self::$standard_remove;
         }
@@ -317,10 +360,11 @@ class Cache extends Getters
         foreach ($remove_paths as $stream) {
 
             // Convert stream to a real path
-            $path = $locator->findResource($stream, true, true);
-            // Make sure path exists before proceeding, otherwise we would wipe ROOT_DIR
-            if (!$path) {
-                throw new \RuntimeException("Stream '{$stream}' not found", 500);
+            try {
+                $path = $locator->findResource($stream, true, true);
+            } catch (\Exception $e) {
+                // stream not found..
+                continue;
             }
 
             $anything = false;

system/src/Grav/Common/Config/Setup.php

@@ -17,6 +17,8 @@ use RocketTheme\Toolbox\ResourceLocator\UniformResourceLocator;
 
 class Setup extends Data
 {
+    public static $environment;
+
     protected $streams = [
         'system' => [
             'type' => 'ReadOnlyStream',
@@ -26,6 +28,7 @@ class Setup extends Data
         ],
         'user' => [
             'type' => 'ReadOnlyStream',
+            'force' => true,
             'prefixes' => [
                 '' => ['user'],
             ]
@@ -78,6 +81,7 @@ class Setup extends Data
         ],
         'cache' => [
             'type' => 'Stream',
+            'force' => true,
             'prefixes' => [
                 '' => ['cache'],
                 'images' => ['images']
@@ -85,16 +89,25 @@ class Setup extends Data
         ],
         'log' => [
             'type' => 'Stream',
+            'force' => true,
             'prefixes' => [
                 '' => ['logs']
             ]
         ],
         'backup' => [
             'type' => 'Stream',
+            'force' => true,
             'prefixes' => [
                 '' => ['backup']
             ]
         ],
+        'tmp' => [
+            'type' => 'Stream',
+            'force' => true,
+            'prefixes' => [
+                '' => ['tmp']
+            ]
+        ],
         'image' => [
             'type' => 'ReadOnlyStream',
             'prefixes' => [
@@ -120,7 +133,7 @@ class Setup extends Data
      */
     public function __construct($container)
     {
-        $environment = $container['uri']->environment() ?: 'localhost';
+        $environment = isset(static::$environment) ? static::$environment : ($container['uri']->environment() ?: 'localhost');
 
         // Pre-load setup.php which contains our initial configuration.
         // Configuration may contain dynamic parts, which is why we need to always load it.
@@ -137,8 +150,8 @@ class Setup extends Data
         parent::__construct($setup);
 
         // Set up environment.
-        $this->def('environment', $environment);
-        $this->def('streams.schemes.environment.prefixes', ['' => ["user://{$this->environment}"]]);
+        $this->def('environment', $environment ?: 'cli');
+        $this->def('streams.schemes.environment.prefixes', ['' => ($environment ? ["user://{$this->environment}"] : [])]);
     }
 
     /**
@@ -194,9 +207,13 @@ class Setup extends Data
             if (isset($config['paths'])) {
                 $locator->addPath($scheme, '', $config['paths']);
             }
+
+            $override = isset($config['override']) ? $config['override'] : false;
+            $force = isset($config['force']) ? $config['force'] : false;
+
             if (isset($config['prefixes'])) {
                 foreach ($config['prefixes'] as $prefix => $paths) {
-                    $locator->addPath($scheme, $prefix, $paths);
+                    $locator->addPath($scheme, $prefix, $paths, $override, $force);
                 }
             }
         }

system/src/Grav/Common/Data/Blueprint.php

@@ -110,6 +110,18 @@ class Blueprint extends BlueprintForm
         return $this->blueprintSchema->filter($data);
     }
 
+    /**
+     * Return blueprint data schema.
+     *
+     * @return BlueprintSchema
+     */
+    public function schema()
+    {
+        $this->initInternals();
+
+        return $this->blueprintSchema;
+    }
+
     /**
      * Initialize validator.
      */

system/src/Grav/Common/Data/Validation.php

@@ -37,12 +37,6 @@ class Validation
             $field['type'] = 'text';
         }
 
-        // Special case for files, value is never empty and errors with code 4 instead.
-        if (empty($validate['required']) && $field['type'] == 'file' && isset($value['error'])
-                && ($value['error'] == UPLOAD_ERR_NO_FILE || in_array(UPLOAD_ERR_NO_FILE, $value['error']))) {
-            return $messages;
-        }
-
         // Get language class.
         $language = Grav::instance()['language'];
 
@@ -101,12 +95,6 @@ class Validation
             $field['type'] = 'text';
         }
 
-        // Special case for files, value is never empty and errors with code 4 instead.
-        if (empty($validate['required']) && $field['type'] == 'file' && isset($value['error'])
-            && ($value['error'] == UPLOAD_ERR_NO_FILE || in_array(UPLOAD_ERR_NO_FILE, $value['error']))) {
-            return null;
-        }
-
         // If this is a YAML field, simply parse it and return the value.
         if (isset($field['yaml']) && $field['yaml'] === true) {
             try {

system/src/Grav/Common/Errors/BareHandler.php

@@ -0,0 +1,24 @@
+<?php
+/**
+ * @package    Grav.Common.Errors
+ *
+ * @copyright  Copyright (C) 2014 - 2016 RocketTheme, LLC. All rights reserved.
+ * @license    MIT License; see LICENSE file for details.
+ */
+
+namespace Grav\Common\Errors;
+
+use Whoops\Handler\Handler;
+
+class BareHandler extends Handler
+{
+
+    /**
+     * @return int|null
+     */
+    public function handle()
+    {
+        return Handler::QUIT;
+    }
+
+}

system/src/Grav/Common/Errors/Errors.php

@@ -22,16 +22,30 @@ class Errors
         // Setup Whoops-based error handler
         $whoops = new \Whoops\Run;
 
+        $verbosity = 1;
+
         if (isset($config['display'])) {
-            if ($config['display']) {
+            if (is_int($config['display'])) {
+                $verbosity = $config['display'];
+            } else {
+                $verbosity = $config['display'] ? 1 : 0;
+            }
+        }
+
+        switch ($verbosity) {
+            case 1:
                 $error_page = new Whoops\Handler\PrettyPageHandler;
                 $error_page->setPageTitle('Crikey! There was an error...');
                 $error_page->addResourcePath(GRAV_ROOT . '/system/assets');
                 $error_page->addCustomCss('whoops.css');
                 $whoops->pushHandler($error_page);
-            } else {
+                break;
+            case -1:
+                $whoops->pushHandler(new BareHandler);
+                break;
+            default:
                 $whoops->pushHandler(new SimplePageHandler);
-            }
+                break;
         }
 
         if (method_exists('Whoops\Util\Misc', 'isAjaxRequest')) { //Whoops 2.0

system/src/Grav/Common/Filesystem/Folder.php

@@ -79,6 +79,34 @@ abstract class Folder
         return $last_modified;
     }
 
+    /**
+     * Recursively md5 hash all files in a path
+     *
+     * @param $path
+     * @return string
+     */
+    public static function hashAllFiles($path)
+    {
+        $flags = \RecursiveDirectoryIterator::SKIP_DOTS;
+        $files = [];
+
+        /** @var UniformResourceLocator $locator */
+        $locator = Grav::instance()['locator'];
+        if ($locator->isStream($path)) {
+            $directory = $locator->getRecursiveIterator($path, $flags);
+        } else {
+            $directory = new \RecursiveDirectoryIterator($path, $flags);
+        }
+
+        $iterator = new \RecursiveIteratorIterator($directory, \RecursiveIteratorIterator::SELF_FIRST);
+
+        foreach ($iterator as $filepath => $file) {
+            $files[] = $file->getPath() . $file->getMTime();
+        }
+
+        return md5(serialize($files));
+    }
+
     /**
      * Get relative path between target and base path. If path isn't relative, return full path.
      *
@@ -179,7 +207,8 @@ abstract class Folder
         /** @var UniformResourceLocator $locator */
         $locator = Grav::instance()['locator'];
         if ($recursive) {
-            $flags = \RecursiveDirectoryIterator::SKIP_DOTS + \FilesystemIterator::UNIX_PATHS + \FilesystemIterator::CURRENT_AS_SELF;
+            $flags = \RecursiveDirectoryIterator::SKIP_DOTS + \FilesystemIterator::UNIX_PATHS
+                + \FilesystemIterator::CURRENT_AS_SELF + \FilesystemIterator::FOLLOW_SYMLINKS;
             if ($locator->isStream($path)) {
                 $directory = $locator->getRecursiveIterator($path, $flags);
             } else {
@@ -402,10 +431,7 @@ abstract class Folder
 
         // If the destination directory does not exist create it
         if (!is_dir($dest)) {
-            if (!mkdir($dest)) {
-                // If the destination directory could not be created stop processing
-                return false;
-            }
+            Folder::mkdir($dest);
         }
 
         // Open the source directory to read in files

system/src/Grav/Common/GPM/GPM.php

@@ -61,14 +61,36 @@ class GPM extends Iterator
     }
 
     /**
-     * Returns the Locally installed packages
-     * @return Iterator The installed packages
+     * Return the locally installed packages
+     *
+     * @return Local\Packages
      */
     public function getInstalled()
     {
         return $this->installed;
     }
 
+    /**
+     * Returns the Locally installable packages
+     *
+     * @param array $list_type_installed
+     * @return Iterator The installed packages
+     */
+    public function getInstallable($list_type_installed = ['plugins' => true, 'themes' => true])
+    {
+        $items = ['total' => 0];
+        foreach ($list_type_installed as $type => $type_installed) {
+            if ($type_installed === false) {
+                continue;
+            }
+            $methodInstallableType = 'getInstalled' . ucfirst($type);
+            $to_install = $this->$methodInstallableType();
+            $items[$type] = $to_install;
+            $items['total'] += count($to_install);
+        }
+        return $items;
+    }
+
     /**
      * Returns the amount of locally installed packages
      * @return integer Amount of installed packages

system/src/Grav/Common/GPM/Installer.php

@@ -29,6 +29,8 @@ class Installer
     const ZIP_OPEN_ERROR = 32;
     /** @const Error while trying to extract the ZIP package */
     const ZIP_EXTRACT_ERROR = 64;
+    /** @const Invalid source file */
+    const INVALID_SOURCE = 128;
 
     /**
      * Destination folder on which validation checks are applied
@@ -62,13 +64,13 @@ class Installer
     /**
      * Installs a given package to a given destination.
      *
-     * @param  string $package     The local path to the ZIP package
+     * @param  string $zip the local path to ZIP package
      * @param  string $destination The local path to the Grav Instance
-     * @param  array  $options     Options to use for installing. ie, ['install_path' => 'user/themes/antimatter']
-     *
-     * @return boolean True if everything went fine, False otherwise.
+     * @param  array $options Options to use for installing. ie, ['install_path' => 'user/themes/antimatter']
+     * @param  string $extracted The local path to the extacted ZIP package
+     * @return bool True if everything went fine, False otherwise.
      */
-    public static function install($package, $destination, $options = [])
+    public static function install($zip, $destination, $options = [], $extracted = null)
     {
         $destination = rtrim($destination, DS);
         $options = array_merge(self::$options, $options);
@@ -86,34 +88,26 @@ class Installer
             return false;
         }
 
-        $zip = new \ZipArchive();
-        $archive = $zip->open($package);
-        $cache_dir = Grav::instance()['locator']->findResource('cache://', true);
-        $tmp = $cache_dir . DS . 'tmp/Grav-' . uniqid();
+        // Create a tmp location
+        $tmp_dir = Grav::instance()['locator']->findResource('tmp://', true, true);
+        $tmp = $tmp_dir . '/Grav-' . uniqid();
 
-        if ($archive !== true) {
-            self::$error = self::ZIP_OPEN_ERROR;
+        if (!$extracted) {
+            $extracted = self::unZip($zip, $tmp);
+            if (!$extracted) {
+                Folder::delete($tmp);
+                return false;
+            }
+        }
 
+        if (!file_exists($extracted)) {
+            self::$error = self::INVALID_SOURCE;
             return false;
         }
 
-        Folder::mkdir($tmp);
-
-        $unzip = $zip->extractTo($tmp);
-
-        if (!$unzip) {
-            self::$error = self::ZIP_EXTRACT_ERROR;
-            $zip->close();
-            Folder::delete($tmp);
-
-            return false;
-        }
-
-        $package_folder_name = $zip->getNameIndex(0);
-        $installer_file_folder = $tmp . '/' . $package_folder_name;
 
         $is_install = true;
-        $installer = self::loadInstaller($installer_file_folder, $is_install);
+        $installer = self::loadInstaller($extracted, $is_install);
 
         if (isset($options['is_update']) && $options['is_update'] === true) {
             $method = 'preUpdate';
@@ -135,16 +129,15 @@ class Installer
 
         if (!$options['sophisticated']) {
             if ($options['theme']) {
-                self::copyInstall($zip, $install_path, $tmp);
+                self::copyInstall($extracted, $install_path);
             } else {
-                self::moveInstall($zip, $install_path, $tmp);
+                self::moveInstall($extracted, $install_path);
             }
         } else {
-            self::sophisticatedInstall($zip, $install_path, $tmp);
+            self::sophisticatedInstall($extracted, $install_path);
         }
 
         Folder::delete($tmp);
-        $zip->close();
 
         if (isset($options['is_update']) && $options['is_update'] === true) {
             $method = 'postUpdate';
@@ -163,6 +156,43 @@ class Installer
 
     }
 
+    /**
+     * Unzip a file to somewhere
+     *
+     * @param $zip_file
+     * @param $destination
+     * @return bool|string
+     */
+    public static function unZip($zip_file, $destination)
+    {
+        $zip = new \ZipArchive();
+        $archive = $zip->open($zip_file);
+
+        if ($archive === true) {
+            Folder::mkdir($destination);
+
+            $unzip = $zip->extractTo($destination);
+
+
+            if (!$unzip) {
+                self::$error = self::ZIP_EXTRACT_ERROR;
+                Folder::delete($destination);
+                $zip->close();
+                return false;
+            }
+
+            $package_folder_name = $zip->getNameIndex(0);
+            $zip->close();
+            $extracted_folder = $destination . '/' . $package_folder_name;
+
+            return $extracted_folder;
+        }
+
+        self::$error = self::ZIP_EXTRACT_ERROR;
+        return false;
+    }
+
+
     /**
      * Instantiates and returns the package installer class
      *
@@ -211,80 +241,67 @@ class Installer
     }
 
     /**
-     * @param \ZipArchive $zip
+     * @param             $source_path
      * @param             $install_path
-     * @param             $tmp
      *
      * @return bool
      */
-    public static function moveInstall(\ZipArchive $zip, $install_path, $tmp)
+    public static function moveInstall($source_path, $install_path)
     {
-        $container = $zip->getNameIndex(0);
         if (file_exists($install_path)) {
             Folder::delete($install_path);
         }
 
-        Folder::move($tmp . DS . $container, $install_path);
+        Folder::move($source_path, $install_path);
 
         return true;
     }
 
     /**
-     * @param \ZipArchive $zip
+     * @param             $source_path
      * @param             $install_path
-     * @param             $tmp
      *
      * @return bool
      */
-    public static function copyInstall(\ZipArchive $zip, $install_path, $tmp)
+    public static function copyInstall($source_path, $install_path)
     {
-        $firstDir = $zip->getNameIndex(0);
-        if (empty($firstDir)) {
-            throw new \RuntimeException("Directory $firstDir is missing");
+        if (empty($source_path)) {
+            throw new \RuntimeException("Directory $source_path is missing");
         } else {
-            $tmp = realpath($tmp . DS . $firstDir);
-            Folder::rcopy($tmp, $install_path);
+            Folder::rcopy($source_path, $install_path);
         }
 
         return true;
     }
 
     /**
-     * @param \ZipArchive $zip
+     * @param             $source_path
      * @param             $install_path
-     * @param             $tmp
      *
      * @return bool
      */
-    public static function sophisticatedInstall(\ZipArchive $zip, $install_path, $tmp)
+    public static function sophisticatedInstall($source_path, $install_path)
     {
-        for ($i = 0, $l = $zip->numFiles; $i < $l; $i++) {
-            $filename = $zip->getNameIndex($i);
-            $fileinfo = pathinfo($filename);
-            $depth = count(explode(DS, rtrim($filename, '/')));
+        foreach (new \DirectoryIterator($source_path) as $file) {
 
-            if ($depth > 2) {
+            if ($file->isLink() || $file->isDot()) {
                 continue;
             }
 
-            $path = $install_path . DS . $fileinfo['basename'];
+            $path = $install_path . DS . $file->getBasename();
 
-            if (is_link($path)) {
-                continue;
-            } else {
-                if (is_dir($path)) {
-                    Folder::delete($path);
-                    Folder::move($tmp . DS . $filename, $path);
+            if ($file->isDir()) {
+                Folder::delete($path);
+                Folder::move($file->getPathname(), $path);
 
-                    if ($fileinfo['basename'] == 'bin') {
-                        foreach (glob($path . DS . '*') as $file) {
-                            @chmod($file, 0755);
-                        }
+                if ($file->getBasename() == 'bin') {
+                    foreach (glob($path . DS . '*') as $bin_file) {
+                           @chmod($bin_file, 0755);
                     }
-                } else {
-                    @unlink($path);
-                    @copy($tmp . DS . $filename, $path);
                 }
+            } else {
+                @unlink($path);
+                @copy($file->getPathname(), $path);
             }
         }
 

system/src/Grav/Common/GPM/Licenses.php

@@ -0,0 +1,126 @@
+<?php
+/**
+ * @package    Grav.Common.GPM
+ *
+ * @copyright  Copyright (C) 2014 - 2016 RocketTheme, LLC. All rights reserved.
+ * @license    MIT License; see LICENSE file for details.
+ */
+
+namespace Grav\Common\GPM;
+
+use Grav\Common\File\CompiledYamlFile;
+use Grav\Common\Grav;
+
+/**
+ * Class Licenses
+ *
+ * @package Grav\Common\GPM
+ */
+class Licenses
+{
+
+    /**
+     * Regex to validate the format of a License
+     *
+     * @var string
+     */
+    protected static $regex = '^(?:[A-F0-9]{8}-){3}(?:[A-F0-9]{8}){1}$';
+
+    protected static $file;
+
+
+    /**
+     * Returns the license for a Premium package
+     *
+     * @param $slug
+     * @param $license
+     *
+     * @return boolean
+     */
+    public static function set($slug, $license)
+    {
+        $licenses = self::getLicenseFile();
+        $data = $licenses->content();
+        $slug = strtolower($slug);
+
+        if ($license && !self::validate($license)) {
+            return false;
+        }
+
+        if (!is_string($license)) {
+            if (isset($data['licenses'][$slug])) {
+                unset($data['licenses'][$slug]);
+            } else {
+                return false;
+            }
+        } else {
+            $data['licenses'][$slug] = $license;
+        }
+
+        $licenses->save($data);
+        $licenses->free();
+
+        return true;
+    }
+
+    /**
+     * Returns the license for a Premium package
+     *
+     * @param $slug
+     *
+     * @return string
+     */
+    public static function get($slug = null)
+    {
+        $licenses = self::getLicenseFile();
+        $data = $licenses->content();
+        $licenses->free();
+        $slug = strtolower($slug);
+
+        if (!$slug) {
+            return isset($data['licenses']) ? $data['licenses'] : [];
+        }
+
+        if (!isset($data['licenses']) || !isset($data['licenses'][$slug])) {
+            return '';
+        }
+
+        return $data['licenses'][$slug];
+    }
+
+
+    /**
+     * Validates the License format
+     *
+     * @param $license
+     *
+     * @return bool
+     */
+    public static function validate($license = null)
+    {
+        if (!is_string($license)) {
+            return false;
+        }
+
+        return preg_match('#' . self::$regex. '#', $license);
+    }
+
+    /**
+     * Get's the License File object
+     *
+     * @return \RocketTheme\Toolbox\File\FileInterface
+     */
+    public static function getLicenseFile()
+
+    {
+        if (!isset(self::$file)) {
+            $path = Grav::instance()['locator']->findResource('user://data') . '/licenses.yaml';;
+            if (!file_exists($path)) {
+                touch($path);
+            }
+            self::$file = CompiledYamlFile::instance($path);
+        }
+
+        return self::$file;
+    }
+}

system/src/Grav/Common/GPM/Response.php

@@ -39,8 +39,10 @@ class Response
             CURLOPT_USERAGENT      => 'Grav GPM',
             CURLOPT_RETURNTRANSFER => true,
             CURLOPT_FOLLOWLOCATION => true,
+            CURLOPT_FAILONERROR    => true,
             CURLOPT_TIMEOUT        => 15,
             CURLOPT_HEADER         => false,
+            //CURLOPT_SSL_VERIFYPEER => true, // this is set in the constructor since it's a setting
             /**
              * Example of callback parameters from within your own class
              */
@@ -48,11 +50,17 @@ class Response
             //CURLOPT_PROGRESSFUNCTION => [$this, 'progress']
         ],
         'fopen' => [
-            'method'          => 'GET',
-            'user_agent'      => 'Grav GPM',
-            'max_redirects'   => 5,
-            'follow_location' => 1,
-            'timeout'         => 15,
+            'method'           => 'GET',
+            'user_agent'       => 'Grav GPM',
+            'max_redirects'    => 5,
+            'follow_location'  => 1,
+            'timeout'          => 15,
+            /* // this is set in the constructor since it's a setting
+            'ssl'              => [
+                'verify_peer'      => true,
+                'verify_peer_name' => true,
+            ],
+            */
             /**
              * Example of callback parameters from within your own class
              */
@@ -101,8 +109,61 @@ class Response
         } catch (\Exception $e) {
         }
 
-        $options = array_replace_recursive(self::$defaults, $options);
-        $method  = 'get' . ucfirst(strtolower(self::$method));
+        $config = Grav::instance()['config'];
+        $overrides = [];
+
+        // SSL Verify Peer and Proxy Setting
+        $settings = [
+            'method'      => $config->get('system.gpm.method', self::$method),
+            'verify_peer' => $config->get('system.gpm.verify_peer', true),
+            // `system.proxy_url` is for fallback
+            // introduced with 1.1.0-beta.1 probably safe to remove at some point
+            'proxy_url'   => $config->get('system.gpm.proxy_url', $config->get('system.proxy_url', false)),
+        ];
+
+        if (!$settings['verify_peer']) {
+            $overrides = array_replace_recursive([], $overrides, [
+                'curl' => [
+                    CURLOPT_SSL_VERIFYPEER => $settings['verify_peer']
+                ],
+                'fopen' => [
+                    'ssl' => [
+                        'verify_peer' => $settings['verify_peer'],
+                        'verify_peer_name' => $settings['verify_peer'],
+                    ]
+                ]
+            ]);
+        }
+
+        // Proxy Setting
+        if ($settings['proxy_url']) {
+            $proxy = parse_url($settings['proxy_url']);
+            $fopen_proxy = ($proxy['scheme'] ?: 'http') . '://' . $proxy['host'] . (isset($proxy['port']) ? ':' . $proxy['port'] : '');
+
+            $overrides = array_replace_recursive([], $overrides, [
+                'curl' => [
+                    CURLOPT_PROXY => $proxy['host'],
+                    CURLOPT_PROXYTYPE => 'HTTP'
+                ],
+                'fopen' => [
+                    'proxy'           => $fopen_proxy,
+                    'request_fulluri' => true
+                ]
+            ]);
+
+            if (isset($proxy['port'])) {
+                $overrides['curl'][CURLOPT_PROXYPORT] = $proxy['port'];
+            }
+
+            if (isset($proxy['user']) && isset($proxy['pass'])) {
+                $fopen_auth = $auth = base64_encode($proxy['user'] . ':' . $proxy['pass']);
+                $overrides['curl'][CURLOPT_PROXYUSERPWD] = $proxy['user'] . ':' . $proxy['pass'];
+                $overrides['fopen']['header'] = "Proxy-Authorization: Basic $fopen_auth";
+            }
+        }
+
+        $options = array_replace_recursive(self::$defaults, $options, $overrides);
+        $method  = 'get' . ucfirst(strtolower($settings['method']));
 
         self::$callback = $callback;
         return static::$method($uri, $options, $callback);
@@ -199,30 +260,39 @@ class Response
         $options  = $args[1];
         $callback = $args[2];
 
-        // if proxy set add that
-        $config = Grav::instance()['config'];
-        $proxy_url = $config->get('system.gpm.proxy_url', $config->get('system.proxy_url'));
-        if ($proxy_url) {
-            $parsed_url = parse_url($proxy_url);
-
-            $options['fopen']['proxy'] = ($parsed_url['scheme'] ?: 'http') . '://' . $parsed_url['host'] . (isset($parsed_url['port']) ? ':' . $parsed_url['port'] : '');
-            $options['fopen']['request_fulluri'] = true;
-
-            if (isset($parsed_url['user']) && isset($parsed_url['pass'])) {
-                $auth = base64_encode($parsed_url['user'] . ':' . $parsed_url['pass']);
-                $options['fopen']['header'] = "Proxy-Authorization: Basic $auth";
-            }
-        }
-
         if ($callback) {
             $options['fopen']['notification'] = ['self', 'progress'];
         }
 
-        $stream  = stream_context_create(['http' => $options['fopen']], $options['fopen']);
+        if (isset($options['fopen']['ssl'])) {
+            $ssl = $options['fopen']['ssl'];
+            unset($options['fopen']['ssl']);
+
+            $stream  = stream_context_create([
+                'http' => $options['fopen'],
+                'ssl' => $ssl
+            ], $options['fopen']);
+        } else {
+            $stream  = stream_context_create(['http' => $options['fopen']], $options['fopen']);
+        }
+
+
         $content = @file_get_contents($uri, false, $stream);
 
         if ($content === false) {
-            throw new \RuntimeException("Error while trying to download '$uri'");
+            $code = null;
+            if (isset($http_response_header)) {
+                $code = explode(' ', $http_response_header[0])[1];
+            }
+
+            switch ($code) {
+                case '404':
+                    throw new \RuntimeException("Page not found");
+                case '401':
+                    throw new \RuntimeException("Invalid LICENSE");
+                default:
+                    throw new \RuntimeException("Error while trying to download '$uri'\n");
+            }
         }
 
         return $content;
@@ -248,8 +318,17 @@ class Response
         $errno = curl_errno($ch);
 
         if ($errno) {
-            $error_message = curl_strerror($errno);
-            throw new \RuntimeException("cURL error ({$errno}):\n {$error_message}");
+            $code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
+            $error_message = curl_strerror($errno) . "\n" . curl_error($ch);
+
+            switch ($code) {
+                case '404':
+                    throw new \RuntimeException("Page not found");
+                case '401':
+                    throw new \RuntimeException("Invalid LICENSE");
+                default:
+                    throw new \RuntimeException("Error while trying to download '$uri'\nMessage: $error_message");
+            }
         }
 
         curl_close($ch);
@@ -276,24 +355,6 @@ class Response
             );
         }
 
-        // if proxy set add that
-        $config = Grav::instance()['config'];
-        $proxy_url = $config->get('system.gpm.proxy_url', $config->get('system.proxy_url'));
-        if ($proxy_url) {
-            $parsed_url = parse_url($proxy_url);
-
-            $options['curl'][CURLOPT_PROXY] = $parsed_url['host'];
-            $options['curl'][CURLOPT_PROXYTYPE] = 'HTTP';
-
-            if (isset($parsed_url['port'])) {
-                $options['curl'][CURLOPT_PROXYPORT] = $parsed_url['port'];
-            }
-
-            if (isset($parsed_url['user']) && isset($parsed_url['pass'])) {
-                $options['curl'][CURLOPT_PROXYUSERPWD] = $parsed_url['user'] . ':' . $parsed_url['pass'];
-            }
-        }
-
         // no open_basedir set, we can proceed normally
         if (!ini_get('open_basedir')) {
             curl_setopt_array($ch, $options['curl']);

system/src/Grav/Common/Grav.php

@@ -208,55 +208,17 @@ class Grav extends Container
         }
     }
 
-    /**
-     * Returns mime type for the file format.
-     *
-     * @param string $format
-     *
-     * @return string
-     */
-    public function mime($format)
-    {
-        // look for some standard types
-        switch ($format) {
-            case null:
-                return 'text/html';
-            case 'json':
-                return 'application/json';
-            case 'html':
-                return 'text/html';
-            case 'atom':
-                return 'application/atom+xml';
-            case 'rss':
-                return 'application/rss+xml';
-            case 'xml':
-                return 'application/xml';
-        }
-
-        // Try finding mime type from media
-        $media_types = $this['config']->get('media.types');
-        if (key_exists($format, $media_types)) {
-            $type = $media_types[$format];
-            if (isset($type['mime'])) {
-                return $type['mime'];
-            }
-        }
-
-        // Can't find the mime type, send as HTML
-        return 'text/html';
-    }
-
     /**
      * Set response header.
      */
     public function header()
     {
-        $extension = $this['uri']->extension();
-
         /** @var Page $page */
         $page = $this['page'];
 
-        header('Content-type: ' . $this->mime($extension));
+        $format = $page->templateFormat();
+
+        header('Content-type: ' . Utils::getMimeByExtension($format, 'text/html'));
 
         // Calculate Expires Headers if set to > 0
         $expires = $page->expires();
@@ -279,7 +241,7 @@ class Grav extends Container
         }
 
         // Set debugger data in headers
-        if (!($extension === null || $extension == 'html')) {
+        if (!($format === null || $format == 'html')) {
             $this['debugger']->enabled(false);
         }
 

system/src/Grav/Common/Helpers/Excerpts.php

@@ -0,0 +1,296 @@
+<?php
+/**
+ * @package    Grav.Common.Helpers
+ *
+ * @copyright  Copyright (C) 2014 - 2016 RocketTheme, LLC. All rights reserved.
+ * @license    MIT License; see LICENSE file for details.
+ */
+
+namespace Grav\Common\Helpers;
+
+use Grav\Common\Grav;
+use Grav\Common\Uri;
+use Grav\Common\Utils;
+use Grav\Common\Page\Medium\Medium;
+use RocketTheme\Toolbox\Event\Event;
+
+class Excerpts
+{
+    /**
+     * Process Grav image media URL from HTML tag
+     *
+     * @param $html         HTML tag e.g. `<img src="image.jpg" />`
+     * @param $page         The current page object
+     * @return string       Returns final HTML string
+     */
+    public static function processImageHtml($html, $page)
+    {
+        $excerpt = static::getExcerptFromHtml($html, 'img');
+
+        $original_src = $excerpt['element']['attributes']['src'];
+        $excerpt['element']['attributes']['href'] = $original_src;
+
+        $excerpt = static::processLinkExcerpt($excerpt, $page, 'image');
+
+        $excerpt['element']['attributes']['src'] = $excerpt['element']['attributes']['href'];
+        unset ($excerpt['element']['attributes']['href']);
+
+        $excerpt = static::processImageExcerpt($excerpt, $page);
+
+        $excerpt['element']['attributes']['data-src'] = $original_src;
+
+        $html = static::getHtmlFromExcerpt($excerpt);
+
+        return $html;
+    }
+
+    /**
+     * Get an Excerpt array from a chunk of HTML
+     *
+     * @param $html         Chunk of HTML
+     * @param $tag          a tag, for example `img`
+     * @return array|null   returns nested array excerpt
+     */
+    public static function getExcerptFromHtml($html, $tag)
+    {
+        $doc = new \DOMDocument();
+        $doc->loadHtml($html);
+        $images = $doc->getElementsByTagName($tag);
+        $excerpt = null;
+
+        foreach ($images as $image) {
+            $attributes = [];
+            foreach ($image->attributes as $name => $value) {
+                $attributes[$name] = $value->value;
+            }
+            $excerpt = [
+                'element' => [
+                    'name'       => $image->tagName,
+                    'attributes' => $attributes
+                ]
+            ];
+        }
+
+        return $excerpt;
+    }
+
+    /**
+     * Rebuild HTML tag from an excerpt array
+     *
+     * @param $excerpt
+     * @return string
+     */
+    public static function getHtmlFromExcerpt($excerpt)
+    {
+        $element = $excerpt['element'];
+        $html = '<'.$element['name'];
+
+        if (isset($element['attributes'])) {
+            foreach ($element['attributes'] as $name => $value) {
+                if ($value === null) {
+                    continue;
+                }
+                $html .= ' '.$name.'="'.$value.'"';
+            }
+        }
+
+        if (isset($element['text'])) {
+            $html .= '>';
+            $html .= $element['text'];
+            $html .= '</'.$element['name'].'>';
+        } else {
+            $html .= ' />';
+        }
+
+        return $html;
+    }
+
+    /**
+     * Process a Link excerpt
+     *
+     * @param $excerpt
+     * @param $page
+     * @param string $type
+     * @return mixed
+     */
+    public static function processLinkExcerpt($excerpt, $page, $type = 'link')
+    {
+        $url = $excerpt['element']['attributes']['href'];
+
+        $url_parts = parse_url(htmlspecialchars_decode(urldecode($url)));
+
+        // if there is a query, then parse it and build action calls
+        if (isset($url_parts['query'])) {
+            $actions = array_reduce(explode('&', $url_parts['query']), function ($carry, $item) {
+                $parts = explode('=', $item, 2);
+                $value = isset($parts[1]) ? rawurldecode($parts[1]) : true;
+                $carry[$parts[0]] = $value;
+
+                return $carry;
+            }, []);
+
+            // valid attributes supported
+            $valid_attributes = ['rel', 'target', 'id', 'class', 'classes'];
+
+            // Unless told to not process, go through actions
+            if (array_key_exists('noprocess', $actions)) {
+                unset($actions['noprocess']);
+            } else {
+                // loop through actions for the image and call them
+                foreach ($actions as $attrib => $value) {
+                    $key = $attrib;
+
+                    if (in_array($attrib, $valid_attributes)) {
+                        // support both class and classes
+                        if ($attrib == 'classes') {
+                            $attrib = 'class';
+                        }
+                        $excerpt['element']['attributes'][$attrib] = str_replace(',', ' ', $value);
+                        unset($actions[$key]);
+                    }
+                }
+            }
+
+            $url_parts['query'] = http_build_query($actions, null, '&', PHP_QUERY_RFC3986);
+        }
+
+        // if no query elements left, unset query
+        if (empty($url_parts['query'])) {
+            unset ($url_parts['query']);
+        }
+
+        // set path to / if not set
+        if (empty($url_parts['path'])) {
+            $url_parts['path'] = '';
+        }
+
+        // if special scheme, just return
+        if(isset($url_parts['scheme']) && !Utils::startsWith($url_parts['scheme'], 'http')) {
+            return $excerpt;
+        }
+
+        // handle paths and such
+        $url_parts = Uri::convertUrl($page, $url_parts, $type);
+
+        // build the URL from the component parts and set it on the element
+        $excerpt['element']['attributes']['href'] = Uri::buildUrl($url_parts);
+
+        return $excerpt;
+    }
+
+    /**
+     * Process an image excerpt
+     *
+     * @param $excerpt
+     * @param $page
+     * @return mixed
+     */
+    public static function processImageExcerpt($excerpt, $page)
+    {
+        $url = $excerpt['element']['attributes']['src'];
+
+        $url_parts = parse_url(htmlspecialchars_decode(urldecode($url)));
+
+        if (isset($url_parts['scheme']) && !Utils::startsWith($url_parts['scheme'], 'http')) {
+            $stream_path = $url_parts['scheme'] . '://' . $url_parts['host'] . $url_parts['path'];
+            $url_parts['path'] = $stream_path;
+            unset($url_parts['host']);
+            unset($url_parts['scheme']);
+        }
+
+        $this_host = isset($url_parts['host']) && $url_parts['host'] == Grav::instance()['uri']->host();
+
+        // if there is no host set but there is a path, the file is local
+        if ((!isset($url_parts['host']) || $this_host) && isset($url_parts['path'])) {
+
+            $path_parts = pathinfo($url_parts['path']);
+            $media = null;
+
+            // get the local path to page media if possible
+            if ($path_parts['dirname'] == $page->url(false, false, false)) {
+                // get the media objects for this page
+                $media = $page->media();
+            } else {
+                // see if this is an external page to this one
+                $base_url = rtrim(Grav::instance()['base_url_relative'] . Grav::instance()['pages']->base(), '/');
+                $page_route = '/' . ltrim(str_replace($base_url, '', $path_parts['dirname']), '/');
+
+                $ext_page = Grav::instance()['pages']->dispatch($page_route, true);
+                if ($ext_page) {
+                    $media = $ext_page->media();
+                } else {
+                    Grav::instance()->fireEvent('onMediaLocate', new Event(['route' => $page_route, 'media' => &$media]));
+                }
+            }
+
+            // if there is a media file that matches the path referenced..
+            if ($media && isset($media->all()[$path_parts['basename']])) {
+                // get the medium object
+                /** @var Medium $medium */
+                $medium = $media->all()[$path_parts['basename']];
+
+                // Process operations
+                $medium = static::processMediaActions($medium, $url_parts);
+
+                $alt = isset($excerpt['element']['attributes']['alt']) ? $excerpt['element']['attributes']['alt'] : '';
+                $title = isset($excerpt['element']['attributes']['title']) ? $excerpt['element']['attributes']['title'] : '';
+                $class = isset($excerpt['element']['attributes']['class']) ? $excerpt['element']['attributes']['class'] : '';
+                $id = isset($excerpt['element']['attributes']['id']) ? $excerpt['element']['attributes']['id'] : '';
+
+                $excerpt['element'] = $medium->parseDownElement($title, $alt, $class, $id, true);
+
+            } else {
+                // not a current page media file, see if it needs converting to relative
+                $excerpt['element']['attributes']['src'] = Uri::buildUrl($url_parts);
+            }
+        }
+
+        return $excerpt;
+    }
+
+    /**
+     * Process media actions
+     *
+     * @param $medium
+     * @param $url
+     * @return mixed
+     */
+    public static function processMediaActions($medium, $url)
+    {
+        if (!is_array($url)) {
+            $url_parts = parse_url($url);
+        } else {
+            $url_parts = $url;
+        }
+
+        $actions = [];
+
+        // if there is a query, then parse it and build action calls
+        if (isset($url_parts['query'])) {
+            $actions = array_reduce(explode('&', $url_parts['query']), function ($carry, $item) {
+                $parts = explode('=', $item, 2);
+                $value = isset($parts[1]) ? $parts[1] : null;
+                $carry[] = ['method' => $parts[0], 'params' => $value];
+
+                return $carry;
+            }, []);
+        }
+
+        if (Grav::instance()['config']->get('system.images.auto_fix_orientation')) {
+            $actions[] = ['method' => 'fixOrientation', 'params' => ''];
+        }
+
+        // loop through actions for the image and call them
+        foreach ($actions as $action) {
+            $medium = call_user_func_array([$medium, $action['method']],
+                explode(',', $action['params']));
+        }
+
+        if (isset($url_parts['fragment'])) {
+            $medium->urlHash($url_parts['fragment']);
+        }
+
+        return $medium;
+    }
+
+}

system/src/Grav/Common/Helpers/Truncator.php

@@ -8,12 +8,16 @@
 
 namespace Grav\Common\Helpers;
 
+use DOMText;
 use DOMDocument;
+use DOMWordsIterator;
+use DOMLettersIterator;
 
 /**
- * This file is part of urodoz/truncateHTML.
+ * This file is part of https://github.com/Bluetel-Solutions/twig-truncate-extension
  *
- * (c) Albert Lacarta <urodoz@gmail.com>
+ * Copyright (c) 2015 Bluetel Solutions developers@bluetel.co.uk
+ * Copyright (c) 2015 Alex Wilson ajw@bluetel.co.uk
  *
  * For the full copyright and license information, please view the LICENSE
  * file that was distributed with this source code.
@@ -21,181 +25,192 @@ use DOMDocument;
 
 class Truncator {
 
-    public static $default_options = array(
-        'ellipsis' => '…',
-        'break' => ' ',
-        'length_in_chars' => false,
-        'word_safe' => false,
-    );
+    /**
+     * Safely truncates HTML by a given number of words.
+     * @param  string  $html     Input HTML.
+     * @param  integer $limit    Limit to how many words we preserve.
+     * @param  string  $ellipsis String to use as ellipsis (if any).
+     * @return string            Safe truncated HTML.
+     */
+    public static function truncateWords($html, $limit = 0, $ellipsis = "")
+    {
+        if ($limit <= 0) {
+            return $html;
+        }
 
-    // These tags are allowed to have an ellipsis inside
-    public static $ellipsable_tags = array(
-        'p', 'ol', 'ul', 'li',
-        'div', 'header', 'article', 'nav',
-        'section', 'footer', 'aside',
-        'dd', 'dt', 'dl',
-    );
+        $dom = self::htmlToDomDocument($html);
 
-    public static $self_closing_tags = array(
-        'br', 'hr', 'img',
-    );
+        // Grab the body of our DOM.
+        $body = $dom->getElementsByTagName("body")->item(0);
+
+        // Iterate over words.
+        $words = new DOMWordsIterator($body);
+        foreach ($words as $word) {
+
+            // If we have exceeded the limit, we delete the remainder of the content.
+            if ($words->key() >= $limit) {
+
+                // Grab current position.
+                $currentWordPosition = $words->currentWordPosition();
+                $curNode = $currentWordPosition[0];
+                $offset = $currentWordPosition[1];
+                $words = $currentWordPosition[2];
+
+                $curNode->nodeValue = substr(
+                    $curNode->nodeValue,
+                    0,
+                    $words[$offset][1] + strlen($words[$offset][0])
+                );
+
+                self::removeProceedingNodes($curNode, $body);
+
+                if (!empty($ellipsis)) {
+                    self::insertEllipsis($curNode, $ellipsis);
+                }
+
+                break;
+            }
+
+        }
+
+        return self::innerHTML($body);
+    }
 
     /**
-     * Truncate given HTML string to specified length.
-     * If length_in_chars is false it's trimmed by number
-     * of words, otherwise by number of characters.
-     *
-     * @param  string        $html
-     * @param  integer       $length
-     * @param  string|array  $opts
-     * @return string
+     * Safely truncates HTML by a given number of letters.
+     * @param  string  $html     Input HTML.
+     * @param  integer $limit    Limit to how many letters we preserve.
+     * @param  string  $ellipsis String to use as ellipsis (if any).
+     * @return string            Safe truncated HTML.
      */
-    public static function truncate($html, $length, $opts=array())
+    public static function truncateLetters($html, $limit = 0, $ellipsis = "")
     {
-        if (is_string($opts)) $opts = array('ellipsis' => $opts);
-        $opts = array_merge(static::$default_options, $opts);
-        // wrap the html in case it consists of adjacent nodes like <p>foo</p><p>bar</p>
-        $html = mb_convert_encoding("<div>".$html."</div>", 'HTML-ENTITIES', 'UTF-8');
-
-        $root_node = null;
-        // Parse using HTML5Lib if it's available.
-        if (class_exists('HTML5Lib\\Parser')) {
-            try {
-                $doc = \HTML5Lib\Parser::parse($html);
-                $root_node = $doc->documentElement->lastChild->lastChild;
-            }
-            catch (\Exception $e) {
-                ;
-            }
+        if ($limit <= 0) {
+            return $html;
         }
-        if ($root_node === null) {
-            // HTML5Lib not available so we'll have to use DOMDocument
-            // We'll only be able to parse HTML5 if it's valid XML
-            $doc = new DOMDocument('4.01', 'utf-8');
-            $doc->formatOutput = false;
-            $doc->preserveWhiteSpace = true;
-            // loadHTML will fail with HTML5 tags (article, nav, etc)
-            // so we need to suppress errors and if it fails to parse we
-            // retry with the XML parser instead
-            $prev_use_errors = libxml_use_internal_errors(true);
-            if ($doc->loadHTML($html)) {
-                $root_node = $doc->documentElement->lastChild->lastChild;
-            }
-            else if ($doc->loadXML($html)) {
-                $root_node = $doc->documentElement;
-            }
-            else {
-                libxml_use_internal_errors($prev_use_errors);
-                throw new \RuntimeException;
-            }
-            libxml_use_internal_errors($prev_use_errors);
-        }
-        list($text, $_, $opts) = static::truncateNode($doc, $root_node, $length, $opts);
 
-        $text = mb_substr(mb_substr($text, 0, -6), 5);
+        $dom = self::htmlToDomDocument($html);
 
-        return $text;
-    }
+        // Grab the body of our DOM.
+        $body = $dom->getElementsByTagName("body")->item(0);
 
-    protected static function truncateNode($doc, $node, $length, $opts)
-    {
-        if ($length === 0 && !static::ellipsable($node)) {
-            return array('', 1, $opts);
-        }
-        list($inner, $remaining, $opts) = static::innerTruncate($doc, $node, $length, $opts);
-        if (0 === mb_strlen($inner)) {
-            return array(in_array(mb_strtolower($node->nodeName), static::$self_closing_tags) ? $doc->saveXML($node) : "", $length - $remaining, $opts);
-        }
-        while($node->firstChild) {
-            $node->removeChild($node->firstChild);
-        }
-        $newNode = $doc->createDocumentFragment();
-        // handle the ampersand
-        $newNode->appendXml(static::xmlEscape($inner));
-        $node->appendChild($newNode);
-        return array($doc->saveXML($node), $length - $remaining, $opts);
-    }
+        // Iterate over letters.
+        $letters = new DOMLettersIterator($body);
+        foreach ($letters as $letter) {
 
-    protected static function innerTruncate($doc, $node, $length, $opts)
-    {
-        $inner = '';
-        $remaining = $length;
-        foreach($node->childNodes as $childNode) {
-            if ($childNode->nodeType === XML_ELEMENT_NODE) {
-                list($txt, $nb, $opts) = static::truncateNode($doc, $childNode, $remaining, $opts);
-            }
-            else if ($childNode->nodeType === XML_TEXT_NODE) {
-                list($txt, $nb, $opts) = static::truncateText($childNode, $remaining, $opts);
-            } else {
-                $txt = '';
-                $nb  = 0;
-            }
+            // If we have exceeded the limit, we want to delete the remainder of this document.
+            if ($letters->key() >= $limit) {
 
-            // unhandle the ampersand
-            $txt = static::xmlUnescape($txt);
+                $currentText = $letters->currentTextPosition();
+                $currentText[0]->nodeValue = substr($currentText[0]->nodeValue, 0, $currentText[1] + 1);
+                self::removeProceedingNodes($currentText[0], $body);
 
-            $remaining -= $nb;
-            $inner .= $txt;
-            if ($remaining < 0) {
-                if (static::ellipsable($node)) {
-                    $inner = preg_replace('/(?:[\s\pP]+|(?:&(?:[a-z]+|#[0-9]+);?))*$/u', '', $inner).$opts['ellipsis'];
-                    $opts['ellipsis'] = '';
-                    $opts['was_truncated'] = true;
+                if (!empty($ellipsis)) {
+                    self::insertEllipsis($currentText[0], $ellipsis);
                 }
+
                 break;
             }
         }
-        return array($inner, $remaining, $opts);
+
+        return self::innerHTML($body);
     }
 
-    protected static function truncateText($node, $length, $opts)
+    /**
+     * Builds a DOMDocument object from a string containing HTML.
+     * @param string HTML to load
+     * @returns DOMDocument Returns a DOMDocument object.
+     */
+    public static function htmlToDomDocument($html)
     {
-        $string = $node->textContent;
+        if (!$html) {
+            $html = '<p></p>';
+        }
 
-        if ($opts['length_in_chars']) {
-            $count = mb_strlen($string);
-            if ($count <= $length && $length > 0) {
-                return array($string, $count, $opts);
-            }
-            if ($opts['word_safe']) {
-                if (false !== ($breakpoint = mb_strpos($string, $opts['break'], $length))) {
-                    if ($breakpoint < mb_strlen($string) - 1) {
-                        $string = mb_substr($string, 0, $breakpoint) . $opts['break'];
-                    }
+        // Transform multibyte entities which otherwise display incorrectly.
+        $html = mb_convert_encoding($html, 'HTML-ENTITIES', 'UTF-8');
+
+        // Internal errors enabled as HTML5 not fully supported.
+        libxml_use_internal_errors(true);
+
+        // Instantiate new DOMDocument object, and then load in UTF-8 HTML.
+        $dom = new DOMDocument();
+        $dom->encoding = 'UTF-8';
+        $dom->loadHTML($html);
+
+        return $dom;
+    }
+
+    /**
+     * Removes all nodes after the current node.
+     * @param  DOMNode|DOMElement $domNode
+     * @param  DOMNode|DOMElement $topNode
+     * @return void
+     */
+    private static function removeProceedingNodes($domNode, $topNode)
+    {
+        $nextNode = $domNode->nextSibling;
+
+        if ($nextNode !== null) {
+            self::removeProceedingNodes($nextNode, $topNode);
+            $domNode->parentNode->removeChild($nextNode);
+        } else {
+            //scan upwards till we find a sibling
+            $curNode = $domNode->parentNode;
+            while ($curNode !== $topNode) {
+                if ($curNode->nextSibling !== null) {
+                    $curNode = $curNode->nextSibling;
+                    self::removeProceedingNodes($curNode, $topNode);
+                    $curNode->parentNode->removeChild($curNode);
+                    break;
                 }
-                return array($string, $count, $opts);
+                $curNode = $curNode->parentNode;
             }
-            return array(mb_substr($node->textContent, 0, $length), $count, $opts);
-        }
-        else {
-            preg_match_all('/\s*\S+/', $string, $words);
-            $words = $words[0];
-            $count = count($words);
-            if ($count <= $length && $length > 0) {
-                return array($string, $count, $opts);
-            }
-            return array(implode('', array_slice($words, 0, $length)), $count, $opts);
         }
     }
 
-    protected static function ellipsable($node)
+    /**
+     * Inserts an ellipsis
+     * @param  DOMNode|DOMElement $domNode  Element to insert after.
+     * @param  string             $ellipsis Text used to suffix our document.
+     * @return void
+     */
+    private static function insertEllipsis($domNode, $ellipsis)
     {
-        return ($node instanceof DOMDocument)
-        || in_array(mb_strtolower($node->nodeName), static::$ellipsable_tags)
-            ;
+        $avoid = array('a', 'strong', 'em', 'h1', 'h2', 'h3', 'h4', 'h5'); //html tags to avoid appending the ellipsis to
+
+        if (in_array($domNode->parentNode->nodeName, $avoid) && $domNode->parentNode->parentNode !== null) {
+            // Append as text node to parent instead
+            $textNode = new DOMText($ellipsis);
+
+            if ($domNode->parentNode->parentNode->nextSibling) {
+                $domNode->parentNode->parentNode->insertBefore($textNode, $domNode->parentNode->parentNode->nextSibling);
+            } else {
+                $domNode->parentNode->parentNode->appendChild($textNode);
+            }
+
+        } else {
+            // Append to current node
+            $domNode->nodeValue = rtrim($domNode->nodeValue) . $ellipsis;
+        }
     }
 
-    protected static function xmlEscape($string)
-    {
-        $string = str_replace('&', '&amp;', $string);
-        $string = str_replace('<?', '&lt;?', $string);
-        return $string;
+    /**
+     * Returns the innerHTML of a particular DOMElement
+     *
+     * @param $element
+     * @return string
+     */
+    private static function innerHTML($element) {
+        $innerHTML = "";
+        $children = $element->childNodes;
+        foreach ($children as $child)
+        {
+            $tmp_dom = new DOMDocument();
+            $tmp_dom->appendChild($tmp_dom->importNode($child, true));
+            $innerHTML.=trim($tmp_dom->saveHTML());
+        }
+        return $innerHTML;
     }
 
-    protected static function xmlUnescape($string)
-    {
-        $string = str_replace('&amp;', '&', $string);
-        $string = str_replace('&lt;?', '<?', $string);
-        return $string;
-    }
 }

system/src/Grav/Common/Language/LanguageCodes.php

@@ -14,7 +14,7 @@ class LanguageCodes
         'af'         => [ 'name' => 'Afrikaans',                 'nativeName' => 'Afrikaans' ],
         'ak'         => [ 'name' => 'Akan',                      'nativeName' => 'Akan' ], // unverified native name
         'ast'        => [ 'name' => 'Asturian',                  'nativeName' => 'Asturianu' ],
-        'ar'         => [ 'name' => 'Arabic',                    'nativeName' => 'عربي' ],
+        'ar'         => [ 'name' => 'Arabic',                    'nativeName' => 'عربي', 'orientation' => 'rtl'],
         'as'         => [ 'name' => 'Assamese',                  'nativeName' => 'অসমীয়া' ],
         'be'         => [ 'name' => 'Belarusian',                'nativeName' => 'Беларуская' ],
         'bg'         => [ 'name' => 'Bulgarian',                 'nativeName' => 'Български' ],
@@ -48,7 +48,7 @@ class LanguageCodes
         'es-MX'      => [ 'name' => 'Spanish (Mexico)',          'nativeName' => 'Español (de México)' ],
         'et'         => [ 'name' => 'Estonian',                  'nativeName' => 'Eesti keel' ],
         'eu'         => [ 'name' => 'Basque',                    'nativeName' => 'Euskara' ],
-        'fa'         => [ 'name' => 'Persian',                   'nativeName' => 'فارسی' ],
+        'fa'         => [ 'name' => 'Persian',                   'nativeName' => 'فارسی' , 'orientation' => 'rtl' ],
         'fi'         => [ 'name' => 'Finnish',                   'nativeName' => 'Suomi' ],
         'fj-FJ'      => [ 'name' => 'Fijian',                    'nativeName' => 'Vosa vaka-Viti' ],
         'fr'         => [ 'name' => 'French',                    'nativeName' => 'Français' ],
@@ -62,7 +62,7 @@ class LanguageCodes
         'gl'         => [ 'name' => 'Galician',                  'nativeName' => 'Galego' ],
         'gu'         => [ 'name' => 'Gujarati',                  'nativeName' => 'ગુજરાતી' ],
         'gu-IN'      => [ 'name' => 'Gujarati',                  'nativeName' => 'ગુજરાતી' ],
-        'he'         => [ 'name' => 'Hebrew',                    'nativeName' => 'עברית' ],
+        'he'         => [ 'name' => 'Hebrew',                    'nativeName' => 'עברית', 'orientation' => 'rtl' ],
         'hi'         => [ 'name' => 'Hindi',                     'nativeName' => 'हिन्दी' ],
         'hi-IN'      => [ 'name' => 'Hindi (India)',             'nativeName' => 'हिन्दी (भारत)' ],
         'hr'         => [ 'name' => 'Croatian',                  'nativeName' => 'Hrvatski' ],
@@ -135,7 +135,7 @@ class LanguageCodes
         'tt'         => [ 'name' => 'Tatar',                     'nativeName' => 'Tatarça' ],
         'tt-RU'      => [ 'name' => 'Tatar',                     'nativeName' => 'Tatarça' ],
         'uk'         => [ 'name' => 'Ukrainian',                 'nativeName' => 'Українська' ],
-        'ur'         => [ 'name' => 'Urdu',                      'nativeName' => 'اُردو' ],
+        'ur'         => [ 'name' => 'Urdu',                      'nativeName' => 'اُردو', 'orientation' => 'rtl'  ],
         've'         => [ 'name' => 'Venda',                     'nativeName' => 'Tshivenḓa' ],
         'vi'         => [ 'name' => 'Vietnamese',                'nativeName' => 'Tiếng Việt' ],
         'wo'         => [ 'name' => 'Wolof',                     'nativeName' => 'Wolof' ],
@@ -165,6 +165,24 @@ class LanguageCodes
         }
     }
 
+    public static function getOrientation($code)
+    {
+        if (isset(static::$codes[$code])) {
+            if (isset(static::$codes[$code]['orientation'])) {
+                return static::get($code, 'orientation');
+            }
+        }
+        return 'ltr';
+    }
+
+    public static function isRtl($code)
+    {
+        if (static::getOrientation($code) == 'rtl') {
+            return true;
+        }
+        return false;
+    }
+
     public static function getNames(array $keys)
     {
         $results = [];

system/src/Grav/Common/Markdown/ParsedownGravTrait.php

@@ -9,8 +9,7 @@
 namespace Grav\Common\Markdown;
 
 use Grav\Common\Grav;
-use Grav\Common\Uri;
-use Grav\Common\Utils;
+use Grav\Common\Helpers\Excerpts;
 use RocketTheme\Toolbox\Event\Event;
 
 trait ParsedownGravTrait
@@ -18,13 +17,6 @@ trait ParsedownGravTrait
     /** @var Page $page */
     protected $page;
 
-    /** @var Pages $pages */
-    protected $pages;
-
-    /** @var  Uri $uri */
-    protected $uri;
-
-    protected $pages_dir;
     protected $special_chars;
     protected $twig_link_regex = '/\!*\[(?:.*)\]\((\{([\{%#])\s*(.*?)\s*(?:\2|\})\})\)/';
 
@@ -42,10 +34,7 @@ trait ParsedownGravTrait
         $grav = Grav::instance();
 
         $this->page = $page;
-        $this->pages = $grav['pages'];
-        $this->uri = $grav['uri'];
         $this->BlockTypes['{'] [] = "TwigTag";
-        $this->pages_dir = Grav::instance()['locator']->findResource('page://');
         $this->special_chars = ['>' => 'gt', '<' => 'lt', '"' => 'quot'];
 
         if ($defaults === null) {
@@ -69,16 +58,23 @@ trait ParsedownGravTrait
      */
     public function addBlockType($type, $tag, $continuable = false, $completable = false, $index = null)
     {
+        $block = &$this->unmarkedBlockTypes;
+        if ($type) {
+            if (!isset($this->BlockTypes[$type])) {
+                $this->BlockTypes[$type] = [];
+            }
+            $block = &$this->BlockTypes[$type];
+        }
+
         if (!isset($index)) {
-            $this->BlockTypes[$type] [] = $tag;
+            $block[] = $tag;
         } else {
-            array_splice($this->BlockTypes[$type], $index, 0, $tag);
+            array_splice($block, $index, 0, [$tag]);
         }
 
         if ($continuable) {
             $this->continuable_blocks[] = $tag;
         }
-
         if ($completable) {
             $this->completable_blocks[] = $tag;
         }
@@ -92,10 +88,10 @@ trait ParsedownGravTrait
      */
     public function addInlineType($type, $tag, $index = null)
     {
-        if (!isset($index)) {
+        if (!isset($index) || !isset($this->InlineTypes[$type])) {
             $this->InlineTypes[$type] [] = $tag;
         } else {
-            array_splice($this->InlineTypes[$type], $index, 0, $tag);
+            array_splice($this->InlineTypes[$type], $index, 0, [$tag]);
         }
 
         if (strpos($this->inlineMarkerList, $type) === false) {
@@ -151,7 +147,7 @@ trait ParsedownGravTrait
      *
      * @return $this
      */
-    function setSpecialChars($special_chars)
+    public function setSpecialChars($special_chars)
     {
         $this->special_chars = $special_chars;
 
@@ -203,75 +199,9 @@ trait ParsedownGravTrait
             $excerpt = parent::inlineImage($excerpt);
         }
 
-        // Some stuff we will need
-        $actions = [];
-        $media = null;
-
-        // if this is an image
+        // if this is an image process it
         if (isset($excerpt['element']['attributes']['src'])) {
-            $alt = $excerpt['element']['attributes']['alt'] ?: '';
-            $title = $excerpt['element']['attributes']['title'] ?: '';
-            $class = isset($excerpt['element']['attributes']['class']) ? $excerpt['element']['attributes']['class'] : '';
-            $id = isset($excerpt['element']['attributes']['id']) ? $excerpt['element']['attributes']['id'] : '';
-
-            //get the url and parse it
-            $url = parse_url(htmlspecialchars_decode($excerpt['element']['attributes']['src']));
-
-            $this_host = isset($url['host']) && $url['host'] == $this->uri->host();
-
-            // if there is no host set but there is a path, the file is local
-            if ((!isset($url['host']) || $this_host) && isset($url['path'])) {
-                $path_parts = pathinfo($url['path']);
-
-                // get the local path to page media if possible
-                if ($path_parts['dirname'] == $this->page->url(false, false, false)) {
-                    // get the media objects for this page
-                    $media = $this->page->media();
-                } else {
-                    // see if this is an external page to this one
-                    $base_url = rtrim(Grav::instance()['base_url_relative'] . Grav::instance()['pages']->base(), '/');
-                    $page_route = '/' . ltrim(str_replace($base_url, '', $path_parts['dirname']), '/');
-
-                    $ext_page = $this->pages->dispatch($page_route, true);
-                    if ($ext_page) {
-                        $media = $ext_page->media();
-                    }
-                }
-
-                // if there is a media file that matches the path referenced..
-                if ($media && isset($media->all()[$path_parts['basename']])) {
-                    // get the medium object
-                    $medium = $media->all()[$path_parts['basename']];
-
-                    // if there is a query, then parse it and build action calls
-                    if (isset($url['query'])) {
-                        $url['query'] = htmlspecialchars_decode(urldecode($url['query']));
-                        $actions = array_reduce(explode('&', $url['query']), function ($carry, $item) {
-                            $parts = explode('=', $item, 2);
-                            $value = isset($parts[1]) ? $parts[1] : null;
-                            $carry[] = ['method' => $parts[0], 'params' => $value];
-
-                            return $carry;
-                        }, []);
-                    }
-
-                    // loop through actions for the image and call them
-                    foreach ($actions as $action) {
-                        $medium = call_user_func_array([$medium, $action['method']],
-                            explode(',', urldecode($action['params'])));
-                    }
-
-                    if (isset($url['fragment'])) {
-                        $medium->urlHash($url['fragment']);
-                    }
-
-                    $excerpt['element'] = $medium->parseDownElement($title, $alt, $class, $id, true);
-
-                } else {
-                    // not a current page media file, see if it needs converting to relative
-                    $excerpt['element']['attributes']['src'] = Uri::buildUrl($url);
-                }
-            }
+            $excerpt = Excerpts::processImageExcerpt($excerpt, $this->page);
         }
 
         return $excerpt;
@@ -299,63 +229,7 @@ trait ParsedownGravTrait
 
         // if this is a link
         if (isset($excerpt['element']['attributes']['href'])) {
-            $url = parse_url(htmlspecialchars_decode($excerpt['element']['attributes']['href']));
-
-            // if there is a query, then parse it and build action calls
-            if (isset($url['query'])) {
-                $actions = array_reduce(explode('&', $url['query']), function ($carry, $item) {
-                    $parts = explode('=', $item, 2);
-                    $value = isset($parts[1]) ? rawurldecode($parts[1]) : true;
-                    $carry[$parts[0]] = $value;
-
-                    return $carry;
-                }, []);
-
-                // valid attributes supported
-                $valid_attributes = ['rel', 'target', 'id', 'class', 'classes'];
-
-                // Unless told to not process, go through actions
-                if (array_key_exists('noprocess', $actions)) {
-                    unset($actions['noprocess']);
-                } else {
-                    // loop through actions for the image and call them
-                    foreach ($actions as $attrib => $value) {
-                        $key = $attrib;
-
-                        if (in_array($attrib, $valid_attributes)) {
-                            // support both class and classes
-                            if ($attrib == 'classes') {
-                                $attrib = 'class';
-                            }
-                            $excerpt['element']['attributes'][$attrib] = str_replace(',', ' ', $value);
-                            unset($actions[$key]);
-                        }
-                    }
-                }
-
-                $url['query'] = http_build_query($actions, null, '&', PHP_QUERY_RFC3986);
-            }
-
-            // if no query elements left, unset query
-            if (empty($url['query'])) {
-                unset ($url['query']);
-            }
-
-            // set path to / if not set
-            if (empty($url['path'])) {
-                $url['path'] = '';
-            }
-
-            // if special scheme, just return
-            if(isset($url['scheme']) && !Utils::startsWith($url['scheme'], 'http')) {
-                return $excerpt;
-            }
-
-            // handle paths and such
-            $url = Uri::convertUrl($this->page, $url, $type);
-
-            // build the URL from the component parts and set it on the element
-            $excerpt['element']['attributes']['href'] = Uri::buildUrl($url);
+            $excerpt = Excerpts::processLinkExcerpt($excerpt, $this->page, $type);
         }
 
         return $excerpt;

system/src/Grav/Common/Page/Collection.php

@@ -124,6 +124,24 @@ class Collection extends Iterator
         return !empty($this->items[$offset]) ? $this->pages->get($offset) : null;
     }
 
+    /**
+     * Split collection into array of smaller collections.
+     *
+     * @param $size
+     * @return array|Collection[]
+     */
+    public function batch($size)
+    {
+        $chunks = array_chunk($this->items, $size, true);
+
+        $list = [];
+        foreach ($chunks as $chunk) {
+            $list[] = new static($chunk, $this->params, $this->pages);
+        }
+
+        return $list;
+    }
+
     /**
      * Remove item from the list.
      *

system/src/Grav/Common/Page/Media.php

@@ -74,16 +74,13 @@ class Media extends Getters
             }
 
             // Create the base medium
-            if (!empty($types['base'])) {
+            if (empty($types['base'])) {
+                $max = max(array_keys($types['alternative']));
+                $medium = $types['alternative'][$max]['file'];
+                $medium = MediumFactory::scaledFromMedium($medium, $max, 1)['file'];
+            } else {
                 $medium = MediumFactory::fromFile($types['base']['file']);
                 $medium && $medium->set('size', $types['base']['size']);
-            } else if (!empty($types['alternative'])) {
-                $altMedium = reset($types['alternative']);
-                $ratio = key($types['alternative']);
-
-                $altMedium = $altMedium['file'];
-
-                $medium = MediumFactory::scaledFromMedium($altMedium, $ratio, 1)['file'];
             }
 
             if (empty($medium)) {
@@ -103,10 +100,9 @@ class Media extends Getters
             // Build missing alternatives
             if (!empty($types['alternative'])) {
                 $alternatives = $types['alternative'];
-
                 $max = max(array_keys($alternatives));
 
-                for ($i=2; $i < $max; $i++) {
+                for ($i=$max; $i > 1; $i--) {
                     if (isset($alternatives[$i])) {
                         continue;
                     }
@@ -115,7 +111,9 @@ class Media extends Getters
                 }
 
                 foreach ($types['alternative'] as $ratio => $altMedium) {
-                    $medium->addAlternative($ratio, $altMedium['file']);
+                    if ($altMedium['file'] != $medium) {
+                        $medium->addAlternative($ratio, $altMedium['file']);
+                    }
                 }
             }
 

system/src/Grav/Common/Page/Medium/ImageMedium.php

@@ -17,7 +17,7 @@ class ImageMedium extends Medium
     /**
      * @var array
      */
-    protected $thumbnailTypes = [ 'page', 'media', 'default' ];
+    protected $thumbnailTypes = ['page', 'media', 'default'];
 
     /**
      * @var ImageFile
@@ -58,18 +58,13 @@ class ImageMedium extends Medium
      * @var array
      */
     public static $magic_resize_actions = [
-        'resize' => [ 0, 1 ],
-        'forceResize' => [ 0, 1 ],
-        'cropResize' => [ 0, 1 ],
-        'crop' => [ 0, 1, 2, 3 ],
-        'zoomCrop' => [ 0, 1 ]
+        'resize' => [0, 1],
+        'forceResize' => [0, 1],
+        'cropResize' => [0, 1],
+        'crop' => [0, 1, 2, 3],
+        'zoomCrop' => [0, 1]
     ];
 
-    /**
-     * @var array
-     */
-    protected $derivatives = [];
-
     /**
      * @var string
      */
@@ -197,32 +192,49 @@ class ImageMedium extends Medium
      */
     public function srcset($reset = true)
     {
-        if (empty($this->alternatives) && empty($this->derivatives)) {
+        if (empty($this->alternatives)) {
             if ($reset) {
                 $this->reset();
             }
+
             return '';
         }
 
-        if (!empty($this->derivatives)) {
-          asort($this->derivatives);
-
-          foreach ($this->derivatives as $url => $width) {
-              $srcset[] = $url . ' ' . $width . 'w';
-          }
-
-          $srcset[] = $this->url($reset) . ' ' . $this->get('width') . 'w';
-        }
-        else {
-          $srcset = [ $this->url($reset) . ' ' . $this->get('width') . 'w' ];
-          foreach ($this->alternatives as $ratio => $medium) {
-              $srcset[] = $medium->url($reset) . ' ' . $medium->get('width') . 'w';
-          }
+        $srcset = [];
+        foreach ($this->alternatives as $ratio => $medium) {
+            $srcset[] = $medium->url($reset) . ' ' . $medium->get('width') . 'w';
         }
+        $srcset[] = $this->url($reset) . ' ' . $this->get('width') . 'w';
 
         return implode(', ', $srcset);
     }
 
+    /**
+     * Allows the ability to override the Inmage's Pretty name stored in cache
+     *
+     * @param $name
+     */
+    public function setImagePrettyName($name)
+    {
+        $this->set('prettyname', $name);
+        if ($this->image) {
+            $this->image->setPrettyName($name);
+        }
+    }
+
+    public function getImagePrettyName()
+    {
+        if ($this->get('prettyname')) {
+            return $this->get('prettyname');
+        } else {
+            $basename = $this->get('basename');
+            if (preg_match('/[a-z0-9]{40}-(.*)/', $basename, $matches)) {
+                $basename = $matches[1];
+            }
+            return $basename;
+        }
+    }
+
     /**
      * Generate derivatives
      *
@@ -232,31 +244,53 @@ class ImageMedium extends Medium
      * @return $this
      */
     public function derivatives($min_width, $max_width, $step = 200) {
-      $width = $min_width;
-
-      // Do not upscale images.
-      if ($max_width > $this->get('width')) {
-        $max_width = $this->get('width');
-      }
-
-      while ($width <= $max_width) {
-        $ratio = $width / $this->get('width');
-        $derivative = MediumFactory::scaledFromMedium($this, 1, $ratio);
-        if (is_array($derivative)) {
-          $this->addDerivative($derivative['file']);
+        if (!empty($this->alternatives)) {
+            $max = max(array_keys($this->alternatives));
+            $base = $this->alternatives[$max];
+        } else {
+            $base = $this;
         }
-        $width += $step;
-      }
-      return $this;
-    }
 
-    /**
-     * Add a derivative
-     *
-     * @param  ImageMedium $image
-     */
-    public function addDerivative(ImageMedium $image) {
-      $this->derivatives[$image->url()] = $image->get('width');
+        // Do not upscale images.
+        $max_width = min($max_width, $base->get('width'));
+
+        for ($width = $min_width; $width < $max_width; $width = $width + $step) {
+            // Only generate image alternatives that don't already exist
+            if (array_key_exists((int) $width, $this->alternatives)) {
+                continue;
+            }
+
+            $derivative = MediumFactory::fromFile($base->get('filepath'));
+
+            // It's possible that MediumFactory::fromFile returns null if the
+            // original image file no longer exists and this class instance was
+            // retrieved from the page cache
+            if (isset($derivative)) {
+                $index = 2;
+                $widths = array_keys($this->alternatives);
+                sort($widths);
+
+                foreach ($widths as $i => $key) {
+                    if ($width > $key) {
+                        $index += max($i, 1);
+                    }
+                }
+
+                $basename = preg_replace('/(@\d+x){0,1}$/', "@{$width}w", $base->get('basename'), 1);
+                $derivative->setImagePrettyName($basename);
+
+                $ratio = $base->get('width') / $width;
+                $height = $derivative->get('height') / $ratio;
+
+                $derivative->resize($width, $height);
+                $derivative->set('width', $width);
+                $derivative->set('height', $height);
+
+                $this->addAlternative($ratio, $derivative);
+            }
+        }
+
+        return $this;
     }
 
     /**
@@ -464,6 +498,10 @@ class ImageMedium extends Medium
             call_user_func_array([$this->image, $method], $args);
 
             foreach ($this->alternatives as $ratio => $medium) {
+                if (!$medium->image) {
+                    $medium->image();
+                }
+
                 $args_copy = $args;
 
                 // regular image: resize 400x400 -> 200x200
@@ -499,9 +537,7 @@ class ImageMedium extends Medium
         $this->image = ImageFile::open($file)
             ->setCacheDir($cacheDir)
             ->setActualCacheDir($cacheDir)
-            ->setPrettyName(basename($this->get('basename')));
-
-        $this->filter();
+            ->setPrettyName($this->getImagePrettyName());
 
         return $this;
     }
@@ -517,6 +553,8 @@ class ImageMedium extends Medium
             return parent::path(false);
         }
 
+        $this->filter();
+
         if (isset($this->result)) {
             return $this->result;
         }

system/src/Grav/Common/Page/Medium/Medium.php

@@ -100,7 +100,9 @@ class Medium extends Data implements RenderableInterface
         }
 
         $alternative->set('ratio', $ratio);
-        $this->alternatives[(float) $ratio] = $alternative;
+        $width = $alternative->get('width');
+
+        $this->alternatives[$width] = $alternative;
     }
 
     /**
@@ -426,7 +428,6 @@ class Medium extends Data implements RenderableInterface
      */
     public function id($id)
     {
-        xdebug_break();
         if (is_string($id)) {
             $this->attributes['id'] = trim($id);
         }

system/src/Grav/Common/Page/Medium/MediumFactory.php

@@ -119,23 +119,27 @@ class MediumFactory
         }
 
         $ratio = $to / $from;
-        $width = (int) ($medium->get('width') * $ratio);
-        $height = (int) ($medium->get('height') * $ratio);
+        $width = $medium->get('width') * $ratio;
+        $height = $medium->get('height') * $ratio;
 
-        $basename = $medium->get('basename');
-        $basename = str_replace('@'.$from.'x', '@'.$to.'x', $basename);
+        $prev_basename = $medium->get('basename');
+        $basename = str_replace('@'.$from.'x', '@'.$to.'x', $prev_basename);
 
         $debug = $medium->get('debug');
         $medium->set('debug', false);
+        $medium->setImagePrettyName($basename);
 
         $file = $medium->resize($width, $height)->path();
 
         $medium->set('debug', $debug);
+        $medium->setImagePrettyName($prev_basename);
 
         $size = filesize($file);
 
         $medium = self::fromFile($file);
-        $medium->set('size', $size);
+        if ($medium) {
+            $medium->set('size', $size);
+        }
 
         return ['file' => $medium, 'size' => $size];
     }

system/src/Grav/Common/Page/Medium/StaticImageMedium.php

@@ -23,6 +23,6 @@ class StaticImageMedium extends Medium
     {
         empty($attributes['src']) && $attributes['src'] = $this->url($reset);
 
-        return [ 'name' => 'image', 'attributes' => $attributes ];
+        return [ 'name' => 'img', 'attributes' => $attributes ];
     }
 }

system/src/Grav/Common/Page/Medium/ThumbnailImageMedium.php

@@ -52,12 +52,13 @@ class ThumbnailImageMedium extends ImageMedium
      * @param string $title
      * @param string $alt
      * @param string $class
+     * @param string $id
      * @param bool $reset
      * @return string
      */
-    public function html($title = null, $alt = null, $class = null, $reset = true)
+    public function html($title = null, $alt = null, $class = null, $id = null, $reset = true)
     {
-        return $this->bubble('html', [$title, $alt, $class, $reset]);
+        return $this->bubble('html', [$title, $alt, $class, $id, $reset]);
     }
 
     /**

system/src/Grav/Common/Page/Page.php

@@ -54,6 +54,7 @@ class Page
     protected $routable;
     protected $modified;
     protected $redirect;
+    protected $external_url;
     protected $items;
     protected $header;
     protected $frontmatter;
@@ -84,6 +85,7 @@ class Page
     protected $home_route;
     protected $hide_home_route;
     protected $ssl;
+    protected $template_format;
 
     /**
      * @var Page Unmodified (original) version of the page. Used for copying and moving the page.
@@ -162,8 +164,8 @@ class Page
                     unset($process_fields[$field]);
                 }
             }
-            $text_header = Grav::instance()['twig']->processString(json_encode($process_fields), ['page'=>$this]);
-            $this->header((object) (json_decode($text_header, true) + $ignored_fields));
+            $text_header = Grav::instance()['twig']->processString(json_encode($process_fields), ['page' => $this]);
+            $this->header((object)(json_decode($text_header, true) + $ignored_fields));
         }
     }
 
@@ -364,6 +366,9 @@ class Page
             if (isset($this->header->redirect)) {
                 $this->redirect = trim($this->header->redirect);
             }
+            if (isset($this->header->external_url)) {
+                $this->external_url = trim($this->header->external_url);
+            }
             if (isset($this->header->order_dir)) {
                 $this->order_dir = trim($this->header->order_dir);
             }
@@ -414,7 +419,10 @@ class Page
                 $this->last_modified = (bool)$this->header->last_modified;
             }
             if (isset($this->header->ssl)) {
-                $this->ssl = (bool) $this->header->ssl;
+                $this->ssl = (bool)$this->header->ssl;
+            }
+            if (isset($this->header->template_format)) {
+                $this->template_format = $this->header->template_format;
             }
         }
 
@@ -498,7 +506,9 @@ class Page
             $size = 300;
         }
 
-        return html_entity_decode(Utils::truncateHTML($content, $size));
+        $summary = Utils::truncateHTML($content, $size);
+
+        return html_entity_decode($summary);
     }
 
     /**
@@ -620,6 +630,7 @@ class Page
         if ($this->content === null) {
             $this->content();
         }
+
         return $this->getContentMeta();
     }
 
@@ -638,18 +649,20 @@ class Page
      * Return the whole contentMeta array as it currently stands
      *
      * @param null $name
+     *
      * @return mixed
      */
     public function getContentMeta($name = null)
     {
         if ($name) {
-            if(isset($this->content_meta[$name])) {
+            if (isset($this->content_meta[$name])) {
                 return $this->content_meta[$name];
             } else {
                 return null;
             }
 
         }
+
         return $this->content_meta;
     }
 
@@ -657,6 +670,7 @@ class Page
      * Sets the whole content meta array in one shot
      *
      * @param $content_meta
+     *
      * @return mixed
      */
     public function setContentMeta($content_meta)
@@ -757,9 +771,7 @@ class Page
             return (bool)$this->order();
         }
         if ($name == 'folder') {
-            $regex = '/^[0-9]+\./u';
-
-            return preg_replace($regex, '', $this->folder);
+            return preg_replace(PAGE_ORDER_PREFIX_REGEX, '', $this->folder);
         }
         if ($name == 'name') {
             $language = $this->language() ? '.' . $this->language() : '';
@@ -1094,6 +1106,27 @@ class Page
         return $this->template;
     }
 
+    /**
+     * Allows a page to override the output render format, usually the extension provided
+     * in the URL. (e.g. `html`, `json`, `xml`, etc).
+     *
+     * @param null $var
+     *
+     * @return null
+     */
+    public function templateFormat($var = null)
+    {
+        if ($var !== null) {
+            $this->template_format = $var;
+        }
+
+        if (empty($this->template_format)) {
+            $this->template_format = Grav::instance()['uri']->extension();
+        }
+
+        return $this->template_format;
+    }
+
     /**
      * Gets and sets the extension field.
      *
@@ -1127,7 +1160,8 @@ class Page
 
         // if not set in the page get the value from system config
         if (empty($this->url_extension)) {
-            $this->url_extension = trim(isset($this->header->append_url_extension) ? $this->header->append_url_extension : Grav::instance()['config']->get('system.pages.append_url_extension', false));
+            $this->url_extension = trim(isset($this->header->append_url_extension) ? $this->header->append_url_extension : Grav::instance()['config']->get('system.pages.append_url_extension',
+                false));
         }
 
         return $this->url_extension;
@@ -1204,8 +1238,7 @@ class Page
         if ($this->visible === null) {
             // Set item visibility in menu if folder is different from slug
             // eg folder = 01.Home and slug = Home
-            $regex = '/^[0-9]+\./u';
-            if (preg_match($regex, $this->folder)) {
+            if (preg_match(PAGE_ORDER_PREFIX_REGEX, $this->folder)) {
                 $this->visible = true;
             } else {
                 $this->visible = false;
@@ -1289,7 +1322,7 @@ class Page
     public function ssl($var = null)
     {
         if ($var !== null) {
-            $this->ssl = (bool) $var;
+            $this->ssl = (bool)$var;
         }
 
         return $this->ssl;
@@ -1350,21 +1383,28 @@ class Page
                 // Backward compatibility for nested arrays in metas
                 if (is_array($value)) {
                     foreach ($value as $property => $prop_value) {
-                        $prop_key                  = $key . ":" . $property;
-                        $this->metadata[$prop_key] = ['name' => $prop_key, 'property' => $prop_key, 'content' => htmlspecialchars($prop_value, ENT_QUOTES, 'UTF-8')];
+                        $prop_key = $key . ":" . $property;
+                        $this->metadata[$prop_key] = [
+                            'name'     => $prop_key,
+                            'property' => $prop_key,
+                            'content'  => htmlspecialchars($prop_value, ENT_QUOTES, 'UTF-8')
+                        ];
                     }
                 } else {
                     // If it this is a standard meta data type
                     if ($value) {
                         if (in_array($key, $header_tag_http_equivs)) {
-                            $this->metadata[$key] = ['http_equiv' => $key, 'content' => htmlspecialchars($value, ENT_QUOTES, 'UTF-8')];
+                            $this->metadata[$key] = [
+                                'http_equiv' => $key,
+                                'content'    => htmlspecialchars($value, ENT_QUOTES, 'UTF-8')
+                            ];
                         } elseif ($key == 'charset') {
                             $this->metadata[$key] = ['charset' => htmlspecialchars($value, ENT_QUOTES, 'UTF-8')];
                         } else {
                             // if it's a social metadata with separator, render as property
-                            $separator    = strpos($key, ':');
+                            $separator = strpos($key, ':');
                             $hasSeparator = $separator && $separator < strlen($key) - 1;
-                            $entry        = ['name' => $key, 'content' => htmlspecialchars($value, ENT_QUOTES, 'UTF-8')];
+                            $entry = ['name' => $key, 'content' => htmlspecialchars($value, ENT_QUOTES, 'UTF-8')];
 
                             if ($hasSeparator) {
                                 $entry['property'] = $key;
@@ -1392,8 +1432,8 @@ class Page
     {
         if ($var !== null && $var !== "") {
             $this->slug = $var;
-            if(!preg_match('/^[a-z0-9][-a-z0-9]*$/', $this->slug)){
-                Grav::instance()['log']->notice("Invalid slug set in YAML frontmatter: " . $this->rawRoute() . " => ".  $this->slug);
+            if (!preg_match('/^[a-z0-9][-a-z0-9]*$/', $this->slug)) {
+                Grav::instance()['log']->notice("Invalid slug set in YAML frontmatter: " . $this->rawRoute() . " => " . $this->slug);
             }
         }
 
@@ -1402,7 +1442,6 @@ class Page
         }
 
 
-
         return $this->slug;
     }
 
@@ -1470,6 +1509,11 @@ class Page
         /** @var Uri $uri */
         $uri = $grav['uri'];
 
+        // Override any URL when external_url is set
+        if (isset($this->external_url)) {
+            return $this->external_url;
+        }
+
         // get pre-route
         if ($include_lang && $language->enabled()) {
             $pre_route = $language->getLanguageURLPrefix();
@@ -1566,8 +1610,7 @@ class Page
         if (empty($this->raw_route)) {
             $baseRoute = $this->parent ? (string)$this->parent()->rawRoute() : null;
 
-            $regex = '/^[0-9]+\./u';
-            $slug = preg_replace($regex, '', $this->folder);
+            $slug = preg_replace(PAGE_ORDER_PREFIX_REGEX, '', $this->folder);
 
             $this->raw_route = isset($baseRoute) ? $baseRoute . '/' . $slug : null;
         }
@@ -2144,7 +2187,10 @@ class Page
      */
     public function home()
     {
-        return $this->find('/') == $this;
+        $home = Grav::instance()['config']->get('system.home.alias');
+        $is_home = ($this->route() == $home || $this->rawRoute() == $home);
+
+        return $is_home;
     }
 
     /**
@@ -2226,7 +2272,7 @@ class Page
                             if (empty($page->taxonomy[$taxonomy]) || !in_array(htmlspecialchars_decode($item,
                                     ENT_QUOTES), $page->taxonomy[$taxonomy])
                             ) {
-                                $collection->remove();
+                                $collection->remove($page->path());
                             }
                         }
                     }
@@ -2300,11 +2346,6 @@ class Page
             return new Collection($result);
         }
 
-        // We only evaluate commands which start with @
-        if (empty($cmd) || $cmd[0] != '@') {
-            return $value;
-        }
-
         /** @var Pages $pages */
         $pages = Grav::instance()['pages'];
 
@@ -2315,6 +2356,7 @@ class Page
         $results = new Collection();
 
         switch ($current) {
+            case 'self@':
             case '@self':
                 if (!empty($parts)) {
                     switch ($parts[0]) {
@@ -2351,6 +2393,7 @@ class Page
                 $results = $results->published();
                 break;
 
+            case 'page@':
             case '@page':
                 $page = null;
 
@@ -2366,27 +2409,33 @@ class Page
                 // Handle a @page.descendants
                 if (!empty($parts)) {
                     switch ($parts[0]) {
+                        case 'modular':
+                            $results = new Collection();
+                            $results = $results->addPage($page)->Modular();
+                            break;
+                        case 'page':
                         case 'self':
                             $results = new Collection();
-                            $results = $results->addPage($page);
+                            $results = $results->addPage($page)->nonModular();
                             break;
 
                         case 'descendants':
-                            $results = $pages->all($page)->remove($page->path());
+                            $results = $pages->all($page)->remove($page->path())->nonModular();
                             break;
 
                         case 'children':
-                            $results = $page->children();
+                            $results = $page->children()->nonModular();
                             break;
                     }
                 } else {
-                    $results = $page->children();
+                    $results = $page->children()->nonModular();
                 }
 
-                $results = $results->nonModular()->published();
+                $results = $results->published();
 
                 break;
 
+            case 'root@':
             case '@root':
                 if (!empty($parts) && $parts[0] == 'descendants') {
                     $results = $pages->all($pages->root())->nonModular()->published();
@@ -2395,7 +2444,7 @@ class Page
                 }
                 break;
 
-
+            case 'taxonomy@':
             case '@taxonomy':
                 // Gets a collection of pages by using one of the following formats:
                 // @taxonomy.category: blog

system/src/Grav/Common/Page/Pages.php

@@ -745,16 +745,18 @@ class Pages
             switch (strtolower($config->get('system.cache.check.method', 'file'))) {
                 case 'none':
                 case 'off':
-                    $last_modified = 0;
+                    $hash = 0;
                     break;
                 case 'folder':
-                    $last_modified = Folder::lastModifiedFolder($pages_dir);
+                    $hash = Folder::lastModifiedFolder($pages_dir);
                     break;
+                case 'hash':
+                    $hash = Folder::hashAllFiles($pages_dir);
                 default:
-                    $last_modified = Folder::lastModifiedFile($pages_dir);
+                    $hash = Folder::lastModifiedFile($pages_dir);
             }
 
-            $page_cache_id = md5($pages_dir . $last_modified . $language->getActive() . $config->checksum());
+            $page_cache_id = md5($pages_dir . $hash . $language->getActive() . $config->checksum());
 
             list($this->instances, $this->routes, $this->children, $taxonomy_map, $this->sort) = $cache->fetch($page_cache_id);
             if (!$this->instances) {
@@ -846,14 +848,22 @@ class Pages
         }
 
         $content_exists = false;
-        $pages_found = glob($directory . '/*' . CONTENT_EXT);
+        $pages_found = new \GlobIterator($directory . '/*' . CONTENT_EXT);
+        $page_found = null;
+
         $page_extension = '';
 
-        if ($pages_found) {
+        if ($pages_found && count($pages_found) > 0) {
+
             $page_extensions = $language->getFallbackPageExtensions();
+
             foreach ($page_extensions as $extension) {
                 foreach ($pages_found as $found) {
-                    if (preg_match('/^.*\/[0-9A-Za-z\-\_]+(' . $extension . ')$/', $found)) {
+                    if ($found->isDir()) {
+                        continue;
+                    }
+                    $regex = '/^[^\.]*' . preg_quote($extension) . '$/';
+                    if (preg_match($regex, $found->getFilename())) {
                         $page_found = $found;
                         $page_extension = $extension;
                         break 2;
@@ -863,8 +873,7 @@ class Pages
         }
 
         if ($parent && !empty($page_found)) {
-            $file = new \SplFileInfo($page_found);
-            $page->init($file, $page_extension);
+            $page->init($page_found, $page_extension);
 
             $content_exists = true;
 
@@ -895,6 +904,12 @@ class Pages
                     $last_modified = $modified;
                 }
             } elseif ($file->isDir() && !in_array($file->getFilename(), $this->ignore_folders)) {
+
+                // if folder contains separator, continue
+                if (Utils::contains($file->getFilename(), $config->get('system.param_sep', ':'))) {
+                    continue;
+                }
+
                 if (!$page->path()) {
                     $page->path($file->getPath());
                 }
@@ -1065,12 +1080,12 @@ class Pages
         } else {
             // else just sort the list according to specified key
             if (extension_loaded('intl')) {
-                $locale = setlocale(LC_COLLATE, 0); //`setlocale` with a 0 param returns the current locale set    
-                $col = \Collator::create($locale); 
+                $locale = setlocale(LC_COLLATE, 0); //`setlocale` with a 0 param returns the current locale set
+                $col = \Collator::create($locale);
                 if ($col) {
-                    $col->asort($list, $sort_flags);    
+                    $col->asort($list, $sort_flags);
                 } else {
-                    asort($list, $sort_flags);    
+                    asort($list, $sort_flags);
                 }
             } else {
                 asort($list, $sort_flags);

system/src/Grav/Common/Page/Types.php

@@ -52,6 +52,8 @@ class Types implements \ArrayAccess, \Iterator, \Countable
 
             // Register default by default.
             $this->register('default');
+
+            $this->register('external');
         }
 
         foreach ($this->findBlueprints($uri) as $type => $blueprint) {
@@ -80,8 +82,10 @@ class Types implements \ArrayAccess, \Iterator, \Countable
         }
 
         $modular_uri = rtrim($uri, '/') . '/modular';
-        foreach (Folder::all($modular_uri, $options) as $type) {
-            $this->register('modular/' . $type);
+        if (is_dir($modular_uri)) {
+            foreach (Folder::all($modular_uri, $options) as $type) {
+                $this->register('modular/' . $type);
+            }
         }
     }
 

system/src/Grav/Common/Plugin.php

@@ -98,6 +98,8 @@ class Plugin implements EventSubscriberInterface, \ArrayAccess
     }
 
     /**
+     * Determine if this is running under the admin
+     *
      * @return bool
      */
     public function isAdmin()
@@ -105,6 +107,27 @@ class Plugin implements EventSubscriberInterface, \ArrayAccess
         return Utils::isAdminPlugin();
     }
 
+    /**
+     * Determine if this route is in Admin and active for the plugin
+     *
+     * @param $plugin_route
+     * @return bool
+     */
+    protected function isPluginActiveAdmin($plugin_route)
+    {
+        $should_run = false;
+
+        $uri = $this->grav['uri'];
+
+        if (strpos($uri->path(), $this->config->get('plugins.admin.route') . '/' . $plugin_route) === false) {
+            $should_run = false;
+        } elseif (isset($uri->paths()[1]) && $uri->paths()[1] == $plugin_route) {
+            $should_run = true;
+        }
+
+        return $should_run;
+    }
+
     /**
      * @param array $events
      */

system/src/Grav/Common/Processors/PagesProcessor.php

@@ -8,15 +8,38 @@
 
 namespace Grav\Common\Processors;
 
-class PagesProcessor extends ProcessorBase implements ProcessorInterface {
+use Grav\Common\Page\Page;
+
+class PagesProcessor extends ProcessorBase implements ProcessorInterface
+{
 
     public $id = 'pages';
     public $title = 'Pages';
 
-    public function process() {
-      	$this->container['pages']->init();
-      	$this->container->fireEvent('onPagesInitialized');
-      	$this->container->fireEvent('onPageInitialized');
+    public function process()
+    {
+        // Dump Cache state
+        $this->container['debugger']->addMessage($this->container['cache']->getCacheStatus());
+
+        $this->container['pages']->init();
+        $this->container->fireEvent('onPagesInitialized');
+        $this->container->fireEvent('onPageInitialized');
+
+        /** @var Page $page */
+        $page = $this->container['page'];
+
+        if (!$page->routable()) {
+            // If no page found, fire event
+            $event = $this->container->fireEvent('onPageNotFound');
+
+            if (isset($event->page)) {
+                unset ($this->container['page']);
+                $this->container['page'] = $event->page;
+            } else {
+                throw new \RuntimeException('Page Not Found', 404);
+            }
+        }
+
     }
 
 }

system/src/Grav/Common/Service/OutputServiceProvider.php

@@ -15,8 +15,7 @@ class OutputServiceProvider implements ServiceProviderInterface
 {
     public function register(Container $container) {
         $container['output'] = function ($c) {
-            /** @var Grav $c */
-            return $c['twig']->processSite($c['uri']->extension());
+            return $c['twig']->processSite($c['page']->templateFormat());
         };
     }
 }

system/src/Grav/Common/Service/PageServiceProvider.php

@@ -8,6 +8,7 @@
 
 namespace Grav\Common\Service;
 
+use Grav\Common\Page\Page;
 use Pimple\Container;
 use Pimple\ServiceProviderInterface;
 
@@ -37,7 +38,7 @@ class PageServiceProvider implements ServiceProviderInterface
                     if (!isset($_SERVER['HTTPS']) || $_SERVER["HTTPS"] != "on") {
                         $url = "https://" . $_SERVER["HTTP_HOST"] . $_SERVER["REQUEST_URI"];
                         $c->redirect($url);
-                    }                    
+                    }
                 }
 
                 $url = $page->route();
@@ -77,14 +78,13 @@ class PageServiceProvider implements ServiceProviderInterface
                 // Try fallback URL stuff...
                 $c->fallbackUrl($path);
 
-                // If no page found, fire event
-                $event = $c->fireEvent('onPageNotFound');
-
-                if (isset($event->page)) {
-                    $page = $event->page;
-                } else {
-                    throw new \RuntimeException('Page Not Found', 404);
+                if (!$page) {
+                    $path = $c['locator']->findResource('system://pages/notfound.md');
+                    $page = new Page();
+                    $page->init(new \SplFileInfo($path));
+                    $page->routable(false);
                 }
+
             }
 
             return $page;

system/src/Grav/Common/Session.php

@@ -38,7 +38,10 @@ class Session extends BaseSession
         $base_url = $uri->rootUrl(false);
 
         $session_timeout = $config->get('system.session.timeout', 1800);
-        $session_path = $config->get('system.session.path', '/' . ltrim($base_url, '/'));
+        $session_path = $config->get('system.session.path');
+        if (!$session_path) {
+            $session_path = '/' . ltrim($base_url, '/');
+        }
 
         // Activate admin if we're inside the admin path.
         if ($config->get('plugins.admin.enabled')) {
@@ -56,19 +59,30 @@ class Session extends BaseSession
         }
 
         if ($config->get('system.session.enabled') || $is_admin) {
-            // Define session service.
-            parent::__construct($session_timeout, $session_path);
-
             $domain = $uri->host();
             if ($domain === 'localhost') {
                 $domain = '';
             }
+
+            // Fix for HUGE session timeouts
+            if ($session_timeout > 99999999999) {
+                $session_timeout = 9999999999;
+            }
+
+            // Define session service.
+            parent::__construct($session_timeout, $session_path, $domain);
+
             $secure = $config->get('system.session.secure', false);
             $httponly = $config->get('system.session.httponly', true);
 
             $unique_identifier = GRAV_ROOT;
             $inflector = new Inflector();
-            $this->setName($inflector->hyphenize($config->get('system.session.name', 'grav_site')) . '-' . substr(md5($unique_identifier), 0, 7) . ($is_admin ? '-admin' : ''));
+            $session_name = $inflector->hyphenize($config->get('system.session.name', 'grav_site')) . '-' . substr(md5($unique_identifier), 0, 7);
+            $split_session = $config->get('system.session.split', true);
+            if ($is_admin && $split_session) {
+              $session_name .= '-admin';
+            }
+            $this->setName($session_name);
             $this->start();
             setcookie(session_name(), session_id(), time() + $session_timeout, $session_path, $domain, $secure, $httponly);
         }

system/src/Grav/Common/Twig/Twig.php

@@ -83,9 +83,9 @@ class Twig
 
             $active_language = $language->getActive();
 
-            $language_append = '';
+            $path_append = rtrim($this->grav['pages']->base(), '/');
             if ($language->getDefault() != $active_language || $config->get('system.languages.include_default_lang') === true) {
-                $language_append = $active_language ? '/' . $active_language : '';
+                $path_append .= $active_language ? '/' . $active_language : '';
             }
 
             // handle language templates if available
@@ -157,10 +157,10 @@ class Twig
                     'config'            => $config,
                     'uri'               => $this->grav['uri'],
                     'base_dir'          => rtrim(ROOT_DIR, '/'),
-                    'base_url'          => $this->grav['base_url'] . $language_append,
+                    'base_url'          => $this->grav['base_url'] . $path_append,
                     'base_url_simple'   => $this->grav['base_url'],
-                    'base_url_absolute' => $this->grav['base_url_absolute'] . $language_append,
-                    'base_url_relative' => $this->grav['base_url_relative'] . $language_append,
+                    'base_url_absolute' => $this->grav['base_url_absolute'] . $path_append,
+                    'base_url_relative' => $this->grav['base_url_relative'] . $path_append,
                     'theme_dir'         => $locator->findResource('theme://'),
                     'theme_url'         => $this->grav['base_url'] . '/' . $locator->findResource('theme://', false),
                     'site'              => $config->get('site'),

system/src/Grav/Common/Twig/TwigExtension.php

@@ -142,12 +142,16 @@ class TwigExtension extends \Twig_Extension
     public function safeEmailFilter($str)
     {
         $email   = '';
-        $str_len = strlen($str);
-        for ($i = 0; $i < $str_len; $i++) {
-            $email .= "&#" . ord($str[$i]) . ";";
+        for ( $i = 0, $len = strlen( $str ); $i < $len; $i++ ) {
+            $j = rand( 0, 1);
+            if ( $j == 0 ) {
+                $email .= '&#' . ord( $str[$i] ) . ';';
+            } elseif ( $j == 1 ) {
+                $email .= $str[$i];
+            }
         }
 
-        return $email;
+        return str_replace( '@', '&#64;', $email );
     }
 
     /**

system/src/Grav/Common/Uri.php

@@ -8,6 +8,7 @@
 
 namespace Grav\Common;
 
+use Grav\Common\Language\Language;
 use Grav\Common\Page\Page;
 
 class Uri
@@ -207,6 +208,7 @@ class Uri
 
         $grav = Grav::instance();
 
+        /** @var Language $language */
         $language = $grav['language'];
 
         $uri_bits = Uri::parseUrl($url);
@@ -275,15 +277,12 @@ class Uri
         }
 
         // Set some defaults
-        $this->root = $this->base . $this->root_path;
+        $this->root = $grav['config']->get('system.custom_base_url') ?: $this->base . $this->root_path;
         $this->url = $this->base . $this->uri;
 
         // get any params and remove them
         $uri = str_replace($this->root, '', $this->url);
 
-        // remove double slashes
-        $uri = preg_replace('#/{2,}#', '/', $uri);
-
         // remove the setup.php based base if set:
         $setup_base = $grav['pages']->base();
         if ($setup_base) {
@@ -292,7 +291,7 @@ class Uri
 
         // If configured to, redirect trailing slash URI's with a 301 redirect
         if ($config->get('system.pages.redirect_trailing_slash', false) && $uri != '/' && Utils::endsWith($uri, '/')) {
-            $grav->redirect(rtrim($uri, '/'), 301);
+            $grav->redirect(str_replace($this->root, '', rtrim($uri, '/')), 301);
         }
 
         // process params
@@ -344,7 +343,7 @@ class Uri
         }
 
         // Set some Grav stuff
-        $grav['base_url_absolute'] = $this->rootUrl(true);
+        $grav['base_url_absolute'] = $grav['config']->get('system.custom_base_url') ?: $this->rootUrl(true);
         $grav['base_url_relative'] = $this->rootUrl(false);
         $grav['base_url'] = $grav['config']->get('system.absolute_urls') ? $grav['base_url_absolute'] : $grav['base_url_relative'];
     }
@@ -767,14 +766,14 @@ class Uri
     /**
      * Converts links from absolute '/' or relative (../..) to a Grav friendly format
      *
-     * @param Page   $page         the current page to use as reference
+     * @param Page $page the current page to use as reference
      * @param string $url the URL as it was written in the markdown
-     * @param string $type         the type of URL, image | link
-     * @param bool   $absolute     if null, will use system default, if true will use absolute links internally
-     *
+     * @param string $type the type of URL, image | link
+     * @param bool $absolute if null, will use system default, if true will use absolute links internally
+     * @param bool $route_only only return the route, not full URL path
      * @return string the more friendly formatted url
      */
-    public static function convertUrl(Page $page, $url, $type = 'link', $absolute = false)
+    public static function convertUrl(Page $page, $url, $type = 'link', $absolute = false, $route_only = false)
     {
         $grav = Grav::instance();
 
@@ -923,6 +922,10 @@ class Uri
             $url = $url_path;
         }
 
+        if ($route_only) {
+            $url = str_replace($base_url, '', $url);
+        }
+
         return $url;
     }
 

system/src/Grav/Common/User/Group.php

@@ -83,13 +83,13 @@ class Group extends Data
         foreach ($fields as $field) {
             if ($field['type'] == 'text') {
                 $value = $field['name'];
-                if (isset($this->items[$value])) {
-                    $config->set("groups.$this->groupname.$value", $this->items[$value]);
+                if (isset($this->items['data'][$value])) {
+                    $config->set("groups.$this->groupname.$value", $this->items['data'][$value]);
                 }
             }
             if ($field['type'] == 'array') {
                 $value = $field['name'];
-                $arrayValues = Utils::getDotNotation($this->items, $field['name']);
+                $arrayValues = Utils::getDotNotation($this->items['data'], $field['name']);
 
                 if ($arrayValues) {
                     foreach ($arrayValues as $arrayIndex => $arrayValue) {

system/src/Grav/Common/Utils.php

@@ -9,8 +9,6 @@
 namespace Grav\Common;
 
 use DateTime;
-use DateTimeZone;
-use Grav\Common\Grav;
 use Grav\Common\Helpers\Truncator;
 use RocketTheme\Toolbox\Event\Event;
 
@@ -183,26 +181,28 @@ abstract class Utils
      * Truncate HTML by number of characters. not "word-safe"!
      *
      * @param  string $text
-     * @param  int    $length
+     * @param  int $length in characters
+     * @param  string $ellipsis
      *
      * @return string
      */
-    public static function truncateHtml($text, $length = 100)
+    public static function truncateHtml($text, $length = 100, $ellipsis = '...')
     {
-        return Truncator::truncate($text, $length, ['length_in_chars' => true]);
+        return Truncator::truncateLetters($text, $length, $ellipsis);
     }
 
     /**
      * Truncate HTML by number of characters in a "word-safe" manor.
      *
      * @param  string $text
-     * @param  int    $length
+     * @param  int    $length in words
+     * @param  string $ellipsis
      *
      * @return string
      */
-    public static function safeTruncateHtml($text, $length = 100)
+    public static function safeTruncateHtml($text, $length = 25, $ellipsis = '...')
     {
-        return Truncator::truncate($text, $length, ['length_in_chars' => true, 'word_safe' => true]);
+        return Truncator::truncateWords($text, $length, $ellipsis);
     }
 
     /**
@@ -233,7 +233,7 @@ abstract class Utils
             Grav::instance()->fireEvent('onBeforeDownload', new Event(['file' => $file]));
 
             $file_parts = pathinfo($file);
-            $mimetype = Utils::getMimeType($file_parts['extension']);
+            $mimetype = Utils::getMimeByExtension($file_parts['extension']);
             $size   = filesize($file); // File size
 
             // clean all buffers
@@ -319,22 +319,84 @@ abstract class Utils
     }
 
     /**
-     * Return the mimetype based on filename
+     * Return the mimetype based on filename extension
      *
      * @param string $extension Extension of file (eg "txt")
+     * @param string $default
      *
      * @return string
      */
-    public static function getMimeType($extension)
+    public static function getMimeByExtension($extension, $default = 'application/octet-stream')
     {
         $extension = strtolower($extension);
-        $config = Grav::instance()['config']->get('media.types');
 
-        if (isset($config[$extension])) {
-            return $config[$extension]['mime'];
+        // look for some standard types
+        switch ($extension) {
+            case null:
+                return $default;
+            case 'json':
+                return 'application/json';
+            case 'html':
+                return 'text/html';
+            case 'atom':
+                return 'application/atom+xml';
+            case 'rss':
+                return 'application/rss+xml';
+            case 'xml':
+                return 'application/xml';
         }
 
-        return 'application/octet-stream';
+        $media_types = Grav::instance()['config']->get('media.types');
+
+        if (isset($media_types[$extension])) {
+            if (isset($media_types[$extension]['mime'])) {
+                return $media_types[$extension]['mime'];
+            }
+        }
+
+        return $default;
+    }
+
+    /**
+     * Return the mimetype based on filename extension
+     *
+     * @param string $mime mime type (eg "text/html")
+     * @param string $default default value
+     *
+     * @return string
+     */
+    public static function getExtensionByMime($mime, $default = 'html')
+    {
+        $mime = strtolower($mime);
+
+        // look for some standard mime types
+        switch ($mime) {
+            case '*/*':
+            case 'text/*':
+            case 'text/html':
+                return 'html';
+            case 'application/json':
+                return 'json';
+            case 'application/atom+xml':
+                return 'atom';
+            case 'application/rss+xml':
+                return 'rss';
+            case 'application/xml':
+                return 'xml';
+        }
+
+        $media_types = Grav::instance()['config']->get('media.types');
+
+        foreach ($media_types as $extension => $type) {
+            if ($extension == 'defaults') {
+                continue;
+            }
+            if (isset($type['mime']) && $type['mime'] == $mime) {
+                return $extension;
+            }
+        }
+
+        return $default;
     }
 
     /**
@@ -432,6 +494,27 @@ abstract class Utils
         return $result;
     }
 
+    /**
+     * Flatten an array
+     *
+     * @param $array
+     * @return array
+     */
+    public static function arrayFlatten($array)
+    {
+        $flatten = array();
+        foreach ($array as $key => $inner){
+            if (is_array($inner)) {
+                foreach ($inner as $inner_key => $value) {
+                    $flatten[$inner_key] = $value;
+                }
+            } else {
+                $flatten[$key] = $inner;
+            }
+        }
+        return $flatten;
+    }
+
     /**
      * Checks if the passed path contains the language code prefix
      *
@@ -690,12 +773,14 @@ abstract class Utils
      * Set portion of array (passed by reference) for a dot-notation key
      * and set the value
      *
-     * @param $array
-     * @param $key
-     * @param $value
+     * @param      $array
+     * @param      $key
+     * @param      $value
+     * @param bool $merge
+     *
      * @return mixed
      */
-    public static function setDotNotation(&$array, $key, $value)
+    public static function setDotNotation(&$array, $key, $value, $merge = false)
     {
         if (is_null($key)) return $array = $value;
 
@@ -713,7 +798,14 @@ abstract class Utils
             $array =& $array[$key];
         }
 
-        $array[array_shift($keys)] = $value;
+        $key = array_shift($keys);
+
+        if (!$merge || !isset($array[$key])) {
+            $array[$key] = $value;
+        } else {
+            $array[$key] = array_merge($array[$key], $value);
+        }
+
 
         return $array;
     }

system/src/Grav/Console/Cli/CleanCommand.php

@@ -94,6 +94,7 @@ class CleanCommand extends Command
         'vendor/ircmaxell/password-compat/version-test.php',
         'vendor/ircmaxell/password-compat/.travis.yml',
         'vendor/ircmaxell/password-compat/test',
+        'vendor/league/climate/composer.json',
         'vendor/matthiasmullie/minify/bin',
         'vendor/matthiasmullie/minify/composer.json',
         'vendor/matthiasmullie/minify/CONTRIBUTING.md',
@@ -124,6 +125,7 @@ class CleanCommand extends Command
         'vendor/rockettheme/toolbox/.travis.yml',
         'vendor/rockettheme/toolbox/composer.json',
         'vendor/rockettheme/toolbox/phpunit.xml',
+        'vendor/seld/cli-prompt/composer.json',
         'vendor/symfony/console/composer.json',
         'vendor/symfony/console/phpunit.xml.dist',
         'vendor/symfony/console/.gitignore',

system/src/Grav/Console/Cli/ClearCacheCommand.php

@@ -27,6 +27,7 @@ class ClearCacheCommand extends ConsoleCommand
             ->addOption('assets-only', null, InputOption::VALUE_NONE, 'If set will remove only assets/*')
             ->addOption('images-only', null, InputOption::VALUE_NONE, 'If set will remove only images/*')
             ->addOption('cache-only', null, InputOption::VALUE_NONE, 'If set will remove only cache/*')
+            ->addOption('tmp-only', null, InputOption::VALUE_NONE, 'If set will remove only tmp/*')
             ->setHelp('The <info>clear-cache</info> deletes all cache files');
     }
 
@@ -55,6 +56,8 @@ class ClearCacheCommand extends ConsoleCommand
             $remove = 'images-only';
         } elseif ($this->input->getOption('cache-only')) {
             $remove = 'cache-only';
+        } elseif ($this->input->getOption('tmp-only')) {
+            $remove = 'tmp-only';
         } else {
             $remove = 'standard';
         }

system/src/Grav/Console/Cli/SandboxCommand.php

@@ -19,15 +19,16 @@ class SandboxCommand extends ConsoleCommand
      * @var array
      */
     protected $directories = [
+        '/assets',
         '/backup',
         '/cache',
-        '/logs',
         '/images',
-        '/assets',
+        '/logs',
+        '/tmp',
         '/user/accounts',
         '/user/config',
-        '/user/pages',
         '/user/data',
+        '/user/pages',
         '/user/plugins',
         '/user/themes',
     ];

system/src/Grav/Console/Gpm/DirectInstallCommand.php

@@ -0,0 +1,374 @@
+<?php
+/**
+ * @package    Grav.Console
+ *
+ * @copyright  Copyright (C) 2014 - 2016 RocketTheme, LLC. All rights reserved.
+ * @license    MIT License; see LICENSE file for details.
+ */
+
+namespace Grav\Console\Gpm;
+
+use Grav\Common\Grav;
+use Grav\Common\Utils;
+use Grav\Common\Filesystem\Folder;
+use Grav\Common\GPM\Installer;
+use Grav\Common\GPM\Response;
+use Grav\Console\ConsoleCommand;
+use Symfony\Component\Console\Input\InputArgument;
+use Symfony\Component\Console\Question\ConfirmationQuestion;
+use Symfony\Component\Yaml\Yaml;
+
+class DirectInstallCommand extends ConsoleCommand
+{
+
+    /**
+     *
+     */
+    protected function configure()
+    {
+        $this
+            ->setName("direct-install")
+            ->setAliases(['directinstall'])
+            ->addArgument(
+                'package-file',
+                InputArgument::REQUIRED,
+                'The local location or remote URL to an installable package file'
+            )
+            ->setDescription("Installs Grav, plugin, or theme directly from a file or a URL")
+            ->setHelp('The <info>direct-install</info> command installs Grav, plugin, or theme directly from a file or a URL');
+    }
+
+    /**
+     * @return int|null|void
+     */
+    protected function serve()
+    {
+        $package_file = $this->input->getArgument('package-file');
+
+        $helper = $this->getHelper('question');
+        $question = new ConfirmationQuestion('Are you sure you want to direct-install <cyan>'.$package_file.'</cyan> [y|N] ', false);
+
+        $answer = $helper->ask($this->input, $this->output, $question);
+
+        if (!$answer) {
+            $this->output->writeln("exiting...");
+            $this->output->writeln('');
+            exit;
+        }
+
+        $tmp_dir = Grav::instance()['locator']->findResource('tmp://', true, true);
+        $tmp_zip = $tmp_dir . '/Grav-' . uniqid();
+
+        $this->output->writeln("");
+        $this->output->writeln("Preparing to install <cyan>" . $package_file . "</cyan>");
+
+
+        if ($this->isRemote($package_file)) {
+            $zip = $this->downloadPackage($package_file, $tmp_zip);
+        } else {
+            $zip = $this->copyPackage($package_file, $tmp_zip);
+        }
+
+        if (file_exists($zip)) {
+            $tmp_source = $tmp_dir . '/Grav-' . uniqid();
+
+            $this->output->write("  |- Extracting package...    ");
+            $extracted = Installer::unZip($zip, $tmp_source);
+
+            if (!$extracted) {
+                $this->output->write("\x0D");
+                $this->output->writeln("  |- Extracting package...    <red>failed</red>");
+                exit;
+            }
+
+            $this->output->write("\x0D");
+            $this->output->writeln("  |- Extracting package...    <green>ok</green>");
+
+            $type = $this->getPackageType($extracted);
+
+            if (!$type) {
+                $this->output->writeln("  '- <red>ERROR: Not a valid Grav package</red>");
+                $this->output->writeln('');
+                exit;
+            }
+
+            $blueprint = $this->getBlueprints($extracted);
+            if ($blueprint) {
+                if (isset($blueprint['dependencies'])) {
+                    $depencencies = [];
+                    foreach ($blueprint['dependencies'] as $dependency) {
+                        if (is_array($dependency) && isset($dependency['name'])) {
+                            $depencencies[] = $dependency['name'];
+                        } else {
+                            $depencencies[] = $dependency;
+                        }
+                    }
+                    $this->output->writeln("  |- Dependencies found...    <cyan>[" . implode(',', $depencencies) . "]</cyan>");
+
+
+
+                    $question = new ConfirmationQuestion("  |  '- Dependencies will not be satisfied. Continue ? [y|N] ", false);
+                    $answer = $helper->ask($this->input, $this->output, $question);
+
+                    if (!$answer) {
+                        $this->output->writeln("exiting...");
+                        $this->output->writeln('');
+                        exit;
+                    }
+                }
+            }
+
+            if ($type == 'grav') {
+
+                $this->output->write("  |- Checking destination...  ");
+                Installer::isValidDestination(GRAV_ROOT . '/system');
+                if (Installer::IS_LINK === Installer::lastErrorCode()) {
+                    $this->output->write("\x0D");
+                    $this->output->writeln("  |- Checking destination...  <yellow>symbolic link</yellow>");
+                    $this->output->writeln("  '- <red>ERROR: symlinks found...</red> <yellow>" . GRAV_ROOT."</yellow>");
+                    $this->output->writeln('');
+                    exit;
+                }
+
+                $this->output->write("\x0D");
+                $this->output->writeln("  |- Checking destination...  <green>ok</green>");
+
+                $this->output->write("  |- Installing package...  ");
+                Installer::install($zip, GRAV_ROOT, ['sophisticated' => true, 'overwrite' => true, 'ignore_symlinks' => true], $extracted);
+            } else {
+                $name = $this->getPackageName($extracted);
+
+                if (!$name) {
+                    $this->output->writeln("<red>ERROR: Name could not be determined.</red> Please specify with --name|-n");
+                    $this->output->writeln('');
+                    exit;
+                }
+
+                $install_path = $this->getInstallPath($type, $name);
+                $is_update = file_exists($install_path);
+
+                $this->output->write("  |- Checking destination...  ");
+
+                Installer::isValidDestination(GRAV_ROOT . DS . $install_path);
+                if (Installer::lastErrorCode() == Installer::IS_LINK) {
+                    $this->output->write("\x0D");
+                    $this->output->writeln("  |- Checking destination...  <yellow>symbolic link</yellow>");
+                    $this->output->writeln("  '- <red>ERROR: symlink found...</red>  <yellow>" . GRAV_ROOT . DS . $install_path . '</yellow>');
+                    $this->output->writeln('');
+                    exit;
+
+                } else {
+                    $this->output->write("\x0D");
+                    $this->output->writeln("  |- Checking destination...  <green>ok</green>");
+                }
+
+                $this->output->write("  |- Installing package...  ");
+
+                Installer::install($zip, GRAV_ROOT, ['install_path' => $install_path, 'theme' => (($type == 'theme')), 'is_update' => $is_update], $extracted);
+            }
+
+            Folder::delete($tmp_source);
+
+            $this->output->write("\x0D");
+
+            if(Installer::lastErrorCode()) {
+                $this->output->writeln("  '- <red>Installation failed or aborted.</red>");
+                $this->output->writeln('');
+            } else {
+                $this->output->writeln("  |- Installing package...    <green>ok</green>");
+                $this->output->writeln("  '- <green>Success!</green>  ");
+                $this->output->writeln('');
+            }
+
+        } else {
+            $this->output->writeln("  '- <red>ERROR: ZIP package could not be found</red>");
+        }
+
+        Folder::delete($tmp_zip);
+
+        // clear cache after successful upgrade
+        $this->clearCache();
+
+        return true;
+
+    }
+
+    /**
+     * Get the install path for a name and a particular type of package
+     *
+     * @param $type
+     * @param $name
+     * @return string
+     */
+    protected function getInstallPath($type, $name)
+    {
+        $locator = Grav::instance()['locator'];
+
+        if ($type == 'theme') {
+            $install_path = $locator->findResource('themes://', false) . DS . $name;
+        } else {
+            $install_path = $locator->findResource('plugins://', false) . DS . $name;
+        }
+        return $install_path;
+    }
+
+    /**
+     * Try to guess the package name from the source files
+     *
+     * @param $source
+     * @return bool|string
+     */
+    protected function getPackageName($source)
+    {
+        foreach (glob($source . "*.yaml") as $filename) {
+            $name = strtolower(basename($filename, '.yaml'));
+            if ($name == 'blueprints') {
+                continue;
+            }
+            return $name;
+        }
+        return false;
+    }
+
+    /**
+     * Try to guess the package type from the source files
+     *
+     * @param $source
+     * @return bool|string
+     */
+    protected function getPackageType($source)
+    {
+        if (
+            file_exists($source . 'system/defines.php') &&
+            file_exists($source . 'system/config/system.yaml')
+        ) {
+            return 'grav';
+        } else {
+            // must have a blueprint
+            if (!file_exists($source . 'blueprints.yaml')) {
+                return false;
+            }
+
+            // either theme or plugin
+            $name = basename($source);
+            if (Utils::contains($name, 'theme')) {
+                return 'theme';
+            } elseif (Utils::contains($name, 'plugin')) {
+                return 'plugin';
+            }
+            foreach (glob($source . "*.php") as $filename) {
+                $contents = file_get_contents($filename);
+                if (Utils::contains($contents, 'Grav\Common\Theme')) {
+                    return 'theme';
+                } elseif (Utils::contains($contents, 'Grav\Common\Plugin')) {
+                    return 'plugin';
+                }
+            }
+
+            // Assume it's a theme
+            return 'theme';
+        }
+    }
+
+    /**
+     * Determine if this is a local or a remote file
+     *
+     * @param $file
+     * @return bool
+     */
+    protected function isRemote($file)
+    {
+        return (bool) filter_var($file, FILTER_VALIDATE_URL);
+    }
+
+    /**
+     * Find/Parse the blueprint file
+     *
+     * @param $source
+     * @return array|bool
+     */
+    protected function getBlueprints($source)
+    {
+        $blueprint_file = $source . 'blueprints.yaml';
+        if (!file_exists($blueprint_file)) {
+            return false;
+        }
+
+        $blueprint = (array)Yaml::parse(file_get_contents($blueprint_file));
+        return $blueprint;
+    }
+
+    /**
+     * Download the zip package via the URL
+     *
+     * @param $package_file
+     * @param $tmp
+     * @return null|string
+     */
+    private function downloadPackage($package_file, $tmp)
+    {
+        $this->output->write("  |- Downloading package...     0%");
+
+        $package = parse_url($package_file);
+
+
+        $filename = basename($package['path']);
+        $output = Response::get($package_file, [], [$this, 'progress']);
+
+        if ($output) {
+            Folder::mkdir($tmp);
+
+            $this->output->write("\x0D");
+            $this->output->write("  |- Downloading package...   100%");
+            $this->output->writeln('');
+
+            file_put_contents($tmp . DS . $filename, $output);
+
+            return $tmp . DS . $filename;
+        }
+
+        return null;
+
+    }
+
+    /**
+     * Copy the local zip package to tmp
+     *
+     * @param $package_file
+     * @param $tmp
+     * @return null|string
+     */
+    private function copyPackage($package_file, $tmp)
+    {
+        $this->output->write("  |- Copying package...         0%");
+
+        $package_file = realpath($package_file);
+
+        if (file_exists($package_file)) {
+            $filename = basename($package_file);
+
+            Folder::mkdir($tmp);
+
+            $this->output->write("\x0D");
+            $this->output->write("  |- Copying package...       100%");
+            $this->output->writeln('');
+
+            copy(realpath($package_file), $tmp . DS . $filename);
+
+            return $tmp . DS . $filename;
+        }
+
+        return null;
+
+    }
+
+    /**
+     * @param $progress
+     */
+    public function progress($progress)
+    {
+        $this->output->write("\x0D");
+        $this->output->write("  |- Downloading package... " . str_pad($progress['percent'], 5, " ",
+                STR_PAD_LEFT) . '%');
+    }
+}

system/src/Grav/Console/Gpm/IndexCommand.php

@@ -9,7 +9,9 @@
 namespace Grav\Console\Gpm;
 
 use Grav\Common\GPM\GPM;
+use Grav\Common\Utils;
 use Grav\Console\ConsoleCommand;
+use League\CLImate\CLImate;
 use Symfony\Component\Console\Input\InputOption;
 
 class IndexCommand extends ConsoleCommand
@@ -100,31 +102,47 @@ class IndexCommand extends ConsoleCommand
     protected function serve()
     {
         $this->options = $this->input->getOptions();
-
         $this->gpm = new GPM($this->options['force']);
-
         $this->displayGPMRelease();
-
         $this->data = $this->gpm->getRepository();
 
         $data = $this->filter($this->data);
 
-        foreach ($data as $type => $packages) {
-            $this->output->writeln("<green>" . ucfirst($type) . "</green> [ " . count($packages) . " ]");
+        $climate = new CLImate;
+        $climate->extend('Grav\Console\TerminalObjects\Table');
 
-            $index    = 0;
+        if (!$data) {
+            $this->output->writeln('No data was found in the GPM repository stored locally.');
+            $this->output->writeln('Please try clearing cache and running the <green>bin/gpm index -f</green> command again');
+            $this->output->writeln('If this doesn\'t work try tweaking your GPM system settings.');
+            $this->output->writeln('');
+            $this->output->writeln('For more help go to:');
+            $this->output->writeln(' -> <yellow>https://learn.getgrav.org/troubleshooting/common-problems#cannot-connect-to-the-gpm</yellow>');
+
+            die;
+        }
+
+        foreach ($data as $type => $packages) {
+            $this->output->writeln("<green>" . strtoupper($type) . "</green> [ " . count($packages) . " ]");
             $packages = $this->sort($packages);
-            foreach ($packages as $slug => $package) {
-                $this->output->writeln(
-                // index
-                    str_pad($index++ + 1, 2, '0', STR_PAD_LEFT) . ". " .
-                    // package name
-                    "<cyan>" . str_pad($package->name, 20) . "</cyan> " .
-                    // slug
-                    "[" . str_pad($slug, 20, ' ', STR_PAD_BOTH) . "] " .
-                    // version details
-                    $this->versionDetails($package)
-                );
+
+            if (!empty($packages)) {
+
+                $table = [];
+                $index    = 0;
+
+                foreach ($packages as $slug => $package) {
+                    $row = [
+                        'Count' => $index++ + 1,
+                        'Name' => "<cyan>" . Utils::truncate($package->name, 20, false, ' ', '...') . "</cyan> ",
+                        'Slug' => $slug,
+                        'Version'=> $this->version($package),
+                        'Installed' => $this->installed($package)
+                    ];
+                    $table[] = $row;
+                }
+
+                $climate->table($table);
             }
 
             $this->output->writeln('');
@@ -143,7 +161,7 @@ class IndexCommand extends ConsoleCommand
      *
      * @return string
      */
-    private function versionDetails($package)
+    private function version($package)
     {
         $list      = $this->gpm->{'getUpdatable' . ucfirst($package->package_type)}();
         $package   = isset($list[$package->slug]) ? $list[$package->slug] : $package;
@@ -154,21 +172,30 @@ class IndexCommand extends ConsoleCommand
 
         if (!$installed || !$updatable) {
             $version   = $installed ? $local->version : $package->version;
-            $installed = !$installed ? ' (<magenta>not installed</magenta>)' : ' (<cyan>installed</cyan>)';
-
-            return str_pad(" [v<green>" . $version . "</green>]", 35) . $installed;
+            return "v<green>" . $version . "</green>";
         }
 
         if ($updatable) {
-            $installed = !$installed ? ' (<magenta>not installed</magenta>)' : ' (<cyan>installed</cyan>)';
-
-            return str_pad(" [v<red>" . $package->version . "</red> <cyan>➜</cyan> v<green>" . $package->available . "</green>]",
-                61) . $installed;
+            return "v<red>" . $package->version . "</red> <cyan>-></cyan> v<green>" . $package->available . "</green>";
         }
 
         return '';
     }
 
+    /**
+     * @param $package
+     *
+     * @return string
+     */
+    private function installed($package)
+    {
+        $package   = isset($list[$package->slug]) ? $list[$package->slug] : $package;
+        $type      = ucfirst(preg_replace("/s$/", '', $package->package_type));
+        $installed = $this->gpm->{'is' . $type . 'Installed'}($package->slug);
+
+        return !$installed ? '<magenta>not installed</magenta>' : '<cyan>installed</cyan>';
+    }
+
     /**
      * @param $data
      *

system/src/Grav/Console/Gpm/InfoCommand.php

@@ -25,6 +25,8 @@ class InfoCommand extends ConsoleCommand
      */
     protected $gpm;
 
+    protected $all_yes;
+
     /**
      *
      */
@@ -60,6 +62,8 @@ class InfoCommand extends ConsoleCommand
     {
         $this->gpm = new GPM($this->input->getOption('force'));
 
+        $this->all_yes = $this->input->getOption('all-yes');
+
         $this->displayGPMRelease();
 
         $foundPackage = $this->gpm->findPackage($this->input->getArgument('package'));
@@ -133,38 +137,34 @@ class InfoCommand extends ConsoleCommand
 
         // display changelog information
         $questionHelper = $this->getHelper('question');
-        $skipPrompt = $this->input->getOption('all-yes');
+        $question = new ConfirmationQuestion("Would you like to read the changelog? [y|N] ",
+            false);
+        $answer = $this->all_yes ? true : $questionHelper->ask($this->input, $this->output, $question);
 
-        if (!$skipPrompt) {
-            $question = new ConfirmationQuestion("Would you like to read the changelog? [y|N] ",
-                false);
-            $answer = $questionHelper->ask($this->input, $this->output, $question);
+        if ($answer) {
+            $changelog = $foundPackage->changelog;
 
-            if ($answer) {
-                $changelog = $foundPackage->changelog;
+            $this->output->writeln("");
+            foreach ($changelog as $version => $log) {
+                $title = $version . ' [' . $log['date'] . ']';
+                $content = preg_replace_callback("/\d\.\s\[\]\(#(.*)\)/", function ($match) {
+                    return "\n" . ucfirst($match[1]) . ":";
+                }, $log['content']);
 
+                $this->output->writeln('<cyan>'.$title.'</cyan>');
+                $this->output->writeln(str_repeat('-', strlen($title)));
+                $this->output->writeln($content);
                 $this->output->writeln("");
-                foreach ($changelog as $version => $log) {
-                    $title = $version . ' [' . $log['date'] . ']';
-                    $content = preg_replace_callback("/\d\.\s\[\]\(#(.*)\)/", function ($match) {
-                        return "\n" . ucfirst($match[1]) . ":";
-                    }, $log['content']);
 
-                    $this->output->writeln('<cyan>'.$title.'</cyan>');
-                    $this->output->writeln(str_repeat('-', strlen($title)));
-                    $this->output->writeln($content);
-                    $this->output->writeln("");
-
-                    $question = new ConfirmationQuestion("Press [ENTER] to continue or [q] to quit ", true);
-                    if (!$questionHelper->ask($this->input, $this->output, $question)) {
-                        break;
-                    }
-                    $this->output->writeln("");
+                $question = new ConfirmationQuestion("Press [ENTER] to continue or [q] to quit ", true);
+                $answer = $this->all_yes ? false : $questionHelper->ask($this->input, $this->output, $question);
+                if (!$answer) {
+                    break;
                 }
+                $this->output->writeln("");
             }
         }
 
-
         $this->output->writeln('');
 
         if ($installed && $updatable) {

system/src/Grav/Console/Gpm/InstallCommand.php

@@ -11,6 +11,7 @@ namespace Grav\Console\Gpm;
 use Grav\Common\Filesystem\Folder;
 use Grav\Common\GPM\GPM;
 use Grav\Common\GPM\Installer;
+use Grav\Common\GPM\Licenses;
 use Grav\Common\GPM\Response;
 use Grav\Common\GPM\Remote\Package as Package;
 use Grav\Common\Grav;
@@ -49,6 +50,8 @@ class InstallCommand extends ConsoleCommand
     /** @var array */
     protected $demo_processing = [];
 
+    protected $all_yes;
+
     /**
      *
      */
@@ -101,6 +104,8 @@ class InstallCommand extends ConsoleCommand
     {
         $this->gpm = new GPM($this->input->getOption('force'));
 
+        $this->all_yes = $this->input->getOption('all-yes');
+
         $this->displayGPMRelease();
 
         $this->destination = realpath($this->input->getOption('destination'));
@@ -139,16 +144,20 @@ class InstallCommand extends ConsoleCommand
         unset($this->data['not_found']);
         unset($this->data['total']);
 
+
         if (isset($this->local_config)) {
             // Symlinks available, ask if Grav should use them
-
             $this->use_symlinks = false;
             $helper = $this->getHelper('question');
             $question = new ConfirmationQuestion('Should Grav use the symlinks if available? [y|N] ', false);
 
-            if ($helper->ask($this->input, $this->output, $question)) {
+            $answer = $this->all_yes ? false : $helper->ask($this->input, $this->output, $question);
+
+            if ($answer) {
                 $this->use_symlinks = true;
             }
+
+
         }
 
         $this->output->writeln('');
@@ -199,8 +208,9 @@ class InstallCommand extends ConsoleCommand
 
                             $helper = $this->getHelper('question');
                             $question = new ConfirmationQuestion("The package <cyan>$package_name</cyan> is already installed, overwrite? [y|N] ", false);
+                            $answer = $this->all_yes ? true : $helper->ask($this->input, $this->output, $question);
 
-                            if ($helper->ask($this->input, $this->output, $question)) {
+                            if ($answer) {
                                 $is_update = true;
                                 $this->processPackage($package, true, $is_update);
                             } else {
@@ -208,8 +218,8 @@ class InstallCommand extends ConsoleCommand
                             }
                         } else {
                             if (Installer::lastErrorCode() == Installer::IS_LINK) {
-                                $this->output->writeln("<red>Cannot overwrite existing symlink</red>");
-                                return false;
+                                $this->output->writeln("<red>Cannot overwrite existing symlink for </red><cyan>$package_name</cyan>");
+                                $this->output->writeln("");
                             }
                         }
                     }
@@ -244,6 +254,11 @@ class InstallCommand extends ConsoleCommand
         $major_version_changed = explode('.', $new_version)[0] !== explode('.', $old_version)[0];
 
         if ($major_version_changed) {
+            if ($this->all_yes) {
+                $this->output->writeln("The package <cyan>$package_name</cyan> will be updated to a new major version <green>$new_version</green>, from <magenta>$old_version</magenta>");
+                return;
+            }
+
             $question = new ConfirmationQuestion("The package <cyan>$package_name</cyan> will be updated to a new major version <green>$new_version</green>, from <magenta>$old_version</magenta>. Be sure to read what changed with the new major release. Continue? [y|N] ", false);
 
             if (!$helper->ask($this->input, $this->output, $question)) {
@@ -297,9 +312,10 @@ class InstallCommand extends ConsoleCommand
                 $questionNoun = 'packages';
             }
 
-            $question = new ConfirmationQuestion("$questionAction $questionArticle $questionNoun? [y|N] ", false);
+            $question = new ConfirmationQuestion("$questionAction $questionArticle $questionNoun? [Y|n] ", true);
+            $answer = $this->all_yes ? true : $helper->ask($this->input, $this->output, $question);
 
-            if ($helper->ask($this->input, $this->output, $question)) {
+            if ($answer) {
                 foreach ($packages as $dependencyName => $dependencyVersion) {
                     $package = $this->gpm->findPackage($dependencyName);
                     $this->processPackage($package, true, ($type == 'update') ? true : false);
@@ -315,10 +331,9 @@ class InstallCommand extends ConsoleCommand
 
     /**
      * @param      $package
-     * @param bool $skip_prompt
-     * @param bool $update      True if the package is an update
+     * @param bool $is_update      True if the package is an update
      */
-    private function processPackage($package, $skip_prompt = false, $is_update = false)
+    private function processPackage($package, $is_update = false)
     {
         if (!$package) {
             $this->output->writeln("<red>Package not found on the GPM!</red>  ");
@@ -333,7 +348,7 @@ class InstallCommand extends ConsoleCommand
             }
         }
 
-        $symlink ? $this->processSymlink($package, $skip_prompt) : $this->processGpm($package, $skip_prompt, $is_update);
+        $symlink ? $this->processSymlink($package) : $this->processGpm($package, $is_update);
 
         $this->processDemo($package);
     }
@@ -369,7 +384,9 @@ class InstallCommand extends ConsoleCommand
             $helper = $this->getHelper('question');
             $question = new ConfirmationQuestion('Do you wish to install this demo content? [y|N] ', false);
 
-            if (!$helper->ask($this->input, $this->output, $question)) {
+            $answer = $this->all_yes ? true : $helper->ask($this->input, $this->output, $question);
+
+            if (!$answer) {
                 $this->output->writeln("  '- <red>Skipped!</red>  ");
                 $this->output->writeln('');
 
@@ -380,8 +397,9 @@ class InstallCommand extends ConsoleCommand
             if (file_exists($demo_dir . DS . 'pages')) {
                 $pages_backup = 'pages.' . date('m-d-Y-H-i-s');
                 $question = new ConfirmationQuestion('This will backup your current `user/pages` folder to `user/' . $pages_backup . '`, continue? [y|N]', false);
+                $answer = $this->all_yes ? true : $helper->ask($this->input, $this->output, $question);
 
-                if (!$helper->ask($this->input, $this->output, $question)) {
+                if (!$answer) {
                     $this->output->writeln("  '- <red>Skipped!</red>  ");
                     $this->output->writeln('');
 
@@ -452,9 +470,8 @@ class InstallCommand extends ConsoleCommand
 
     /**
      * @param      $package
-     * @param bool $skip_prompt
      */
-    private function processSymlink($package, $skip_prompt = false)
+    private function processSymlink($package)
     {
 
         exec('cd ' . $this->destination);
@@ -469,7 +486,7 @@ class InstallCommand extends ConsoleCommand
             $this->output->writeln("<green>ok</green>");
 
             $this->output->write("  |- Checking destination...  ");
-            $checks = $this->checkDestination($package, $skip_prompt);
+            $checks = $this->checkDestination($package);
 
             if (!$checks) {
                 $this->output->writeln("  '- <red>Installation failed or aborted.</red>");
@@ -497,19 +514,29 @@ class InstallCommand extends ConsoleCommand
 
     /**
      * @param      $package
-     * @param bool $skip_prompt
+     * @param bool $is_update
+     *
+     * @return bool
      */
-    private function processGpm($package, $skip_prompt = false, $is_update = false)
+    private function processGpm($package, $is_update = false)
     {
         $version = isset($package->available) ? $package->available : $package->version;
+        $license = Licenses::get($package->slug);
 
         $this->output->writeln("Preparing to install <cyan>" . $package->name . "</cyan> [v" . $version . "]");
 
         $this->output->write("  |- Downloading package...     0%");
-        $this->file = $this->downloadPackage($package);
+        $this->file = $this->downloadPackage($package, $license);
+
+        if (!$this->file) {
+            $this->output->writeln("  '- <red>Installation failed or aborted.</red>");
+            $this->output->writeln('');
+
+            return false;
+        }
 
         $this->output->write("  |- Checking destination...  ");
-        $checks = $this->checkDestination($package, $skip_prompt);
+        $checks = $this->checkDestination($package);
 
         if (!$checks) {
             $this->output->writeln("  '- <red>Installation failed or aborted.</red>");
@@ -530,14 +557,41 @@ class InstallCommand extends ConsoleCommand
     /**
      * @param Package $package
      *
+     * @param string    $license
+     *
      * @return string
      */
-    private function downloadPackage($package)
+    private function downloadPackage($package, $license = null)
     {
-        $cache_dir = Grav::instance()['locator']->findResource('cache://', true);
-        $this->tmp = $cache_dir . DS . 'tmp/Grav-' . uniqid();
+        $tmp_dir = Grav::instance()['locator']->findResource('tmp://', true, true);
+        $this->tmp = $tmp_dir . '/Grav-' . uniqid();
         $filename = $package->slug . basename($package->zipball_url);
-        $output = Response::get($package->zipball_url, [], [$this, 'progress']);
+        $query = '';
+
+        if ($package->premium) {
+            $query = \json_encode(array_merge(
+                $package->premium,
+                [
+                    'slug' => $package->slug,
+                    'filename' => $package->premium['filename'],
+                    'license_key' => $license
+                ]
+            ));
+
+            $query = '?d=' . base64_encode($query);
+        }
+
+        try {
+            $output = Response::get($package->zipball_url . $query, [], [$this, 'progress']);
+        } catch (\Exception $e) {
+            $error = str_replace("\n", "\n  |  '- ", $e->getMessage());
+            $this->output->write("\x0D");
+            // extra white spaces to clear out the buffer properly
+            $this->output->writeln("  |- Downloading package...    <red>error</red>                             ");
+            $this->output->writeln("  |  '- " . $error);
+
+            return false;
+        }
 
         Folder::mkdir($this->tmp);
 
@@ -553,42 +607,19 @@ class InstallCommand extends ConsoleCommand
     /**
      * @param      $package
      *
-     * @param bool $skip_prompt
-     *
      * @return bool
      */
-    private function checkDestination($package, $skip_prompt = false)
+    private function checkDestination($package)
     {
         $question_helper = $this->getHelper('question');
 
-        if (!$skip_prompt) {
-            $skip_prompt = $this->input->getOption('all-yes');
-        }
-
         Installer::isValidDestination($this->destination . DS . $package->install_path);
 
-        if (Installer::lastErrorCode() == Installer::EXISTS) {
-            if (!$skip_prompt) {
-                $this->output->write("\x0D");
-                $this->output->writeln("  |- Checking destination...  <yellow>exists</yellow>");
-
-                $question = new ConfirmationQuestion("  |  '- The package is already installed, do you want to overwrite it? [y|N] ",
-                    false);
-                $answer = $question_helper->ask($this->input, $this->output, $question);
-
-                if (!$answer) {
-                    $this->output->writeln("  |     '- <red>You decided to not overwrite the already installed package.</red>");
-
-                    return false;
-                }
-            }
-        }
-
         if (Installer::lastErrorCode() == Installer::IS_LINK) {
             $this->output->write("\x0D");
             $this->output->writeln("  |- Checking destination...  <yellow>symbolic link</yellow>");
 
-            if ($skip_prompt) {
+            if ($this->all_yes) {
                 $this->output->writeln("  |     '- <yellow>Skipped automatically.</yellow>");
 
                 return false;

system/src/Grav/Console/Gpm/SelfupgradeCommand.php

@@ -49,6 +49,9 @@ class SelfupgradeCommand extends ConsoleCommand
      */
     private $upgrader;
 
+    protected $all_yes;
+    protected $overwrite;
+
     /**
      *
      */
@@ -69,6 +72,12 @@ class SelfupgradeCommand extends ConsoleCommand
                 InputOption::VALUE_NONE,
                 'Assumes yes (or best approach) instead of prompting'
             )
+            ->addOption(
+                'overwrite',
+                'o',
+                InputOption::VALUE_NONE,
+                'Option to overwrite packages if they already exist'
+            )
             ->setDescription("Detects and performs an update of Grav itself when available")
             ->setHelp('The <info>update</info> command updates Grav itself when a new version is available');
     }
@@ -79,6 +88,8 @@ class SelfupgradeCommand extends ConsoleCommand
     protected function serve()
     {
         $this->upgrader = new Upgrader($this->input->getOption('force'));
+        $this->all_yes = $this->input->getOption('all-yes');
+        $this->overwrite = $this->input->getOption('overwrite');
 
         $this->displayGPMRelease();
 
@@ -89,7 +100,6 @@ class SelfupgradeCommand extends ConsoleCommand
         $release = strftime('%c', strtotime($this->upgrader->getReleaseDate()));
 
         if (!$this->upgrader->meetsRequirements()) {
-            $this->output->writeln("");
             $this->output->writeln("<red>ATTENTION:</red>");
             $this->output->writeln("   Grav has increased the minimum PHP requirement.");
             $this->output->writeln("   You are currently running PHP <red>" . PHP_VERSION . "</red>, but PHP <green>" . GRAV_PHP_MIN . "</green> is required.");
@@ -100,21 +110,29 @@ class SelfupgradeCommand extends ConsoleCommand
             exit;
         }
 
-        if (!$this->upgrader->isUpgradable()) {
+        if (!$this->overwrite && !$this->upgrader->isUpgradable()) {
             $this->output->writeln("You are already running the latest version of Grav (v" . $local . ") released on " . $release);
             exit;
         }
 
+        Installer::isValidDestination(GRAV_ROOT . '/system');
+        if (Installer::IS_LINK === Installer::lastErrorCode()) {
+            $this->output->writeln("<red>ATTENTION:</red> Grav is symlinked, cannot upgrade, aborting...");
+            $this->output->writeln('');
+            $this->output->writeln("You are currently running a symbolically linked Grav v" . $local . ". Latest available is v". $remote . ".");
+            exit;
+        }
+
         // not used but preloaded just in case!
         new ArrayInput([]);
 
         $questionHelper = $this->getHelper('question');
-        $skipPrompt = $this->input->getOption('all-yes');
+
 
         $this->output->writeln("Grav v<cyan>$remote</cyan> is now available [release date: $release].");
         $this->output->writeln("You are currently using v<cyan>" . GRAV_VERSION . "</cyan>.");
 
-        if (!$skipPrompt) {
+        if (!$this->all_yes) {
             $question = new ConfirmationQuestion("Would you like to read the changelog before proceeding? [y|N] ",
                 false);
             $answer = $questionHelper->ask($this->input, $this->output, $question);
@@ -177,8 +195,8 @@ class SelfupgradeCommand extends ConsoleCommand
      */
     private function download($package)
     {
-        $cache_dir = Grav::instance()['locator']->findResource('cache://', true);
-        $this->tmp = $cache_dir . DS . 'tmp/Grav-' . uniqid();
+        $tmp_dir = Grav::instance()['locator']->findResource('tmp://', true, true);
+        $this->tmp = $tmp_dir . '/Grav-' . uniqid();
         $output = Response::get($package['download'], [], [$this, 'progress']);
 
         Folder::mkdir($this->tmp);

system/src/Grav/Console/Gpm/UninstallCommand.php

@@ -39,6 +39,10 @@ class UninstallCommand extends ConsoleCommand
      */
     protected $tmp;
 
+    protected $dependencies= [];
+
+    protected $all_yes;
+
     /**
      *
      */
@@ -68,6 +72,8 @@ class UninstallCommand extends ConsoleCommand
     {
         $this->gpm = new GPM();
 
+        $this->all_yes = $this->input->getOption('all-yes');
+
         $packages = array_map('strtolower', $this->input->getArgument('package'));
         $this->data = ['total' => 0, 'not_found' => []];
 
@@ -108,15 +114,12 @@ class UninstallCommand extends ConsoleCommand
                 $this->output->writeln("  '- <red>Installation failed or aborted.</red>");
                 $this->output->writeln('');
             } else {
-                $this->output->write("  |- Uninstalling package...  ");
                 $uninstall = $this->uninstallPackage($slug, $package);
 
                 if (!$uninstall) {
                     $this->output->writeln("  '- <red>Uninstallation failed or aborted.</red>");
-                    $this->output->writeln('');
                 } else {
                     $this->output->writeln("  '- <green>Success!</green>  ");
-                    $this->output->writeln('');
                 }
             }
 
@@ -156,36 +159,29 @@ class UninstallCommand extends ConsoleCommand
             return false;
         }
 
-        $locator = Grav::instance()['locator'];
-        $path = $locator->findResource($package->package_type . '://' . $slug);
-        Installer::uninstall($path);
-        $errorCode = Installer::lastErrorCode();
-
-        if ($errorCode && $errorCode !== Installer::IS_LINK && $errorCode !== Installer::EXISTS) {
-            $this->output->write("\x0D");
-            // extra white spaces to clear out the buffer properly
-            $this->output->writeln("  |- Uninstalling package...  <red>error</red>                             ");
-            $this->output->writeln("  |  '- " . Installer::lastErrorMsg());
-
-            return false;
-        }
-
-        $message = Installer::getMessage();
-        if ($message) {
-            $this->output->write("\x0D");
-            // extra white spaces to clear out the buffer properly
-            $this->output->writeln("  |- " . $message);
-        }
-
-        $this->output->write("\x0D");
-        // extra white spaces to clear out the buffer properly
-        $this->output->writeln("  |- Uninstalling package...  <green>ok</green>                             ");
-
-
         if (isset($package->dependencies)) {
+
+            $dependencies = $package->dependencies;
+
+            if ($is_dependency) {
+                foreach ($dependencies as $key => $dependency) {
+                    if (in_array($dependency['name'], $this->dependencies)) {
+                        unset($dependencies[$key]);
+                    }
+                }
+            } else {
+                if (count($dependencies) > 0) {
+                    $this->output->writeln('  `- Dependencies found...');
+                    $this->output->writeln('');
+                }
+            }
+
             $questionHelper = $this->getHelper('question');
 
-            foreach($package->dependencies as $dependency) {
+            foreach ($dependencies as $dependency) {
+
+                $this->dependencies[] = $dependency['name'];
+
                 if (is_array($dependency)) {
                     $dependency = $dependency['name'];
                 }
@@ -194,25 +190,59 @@ class UninstallCommand extends ConsoleCommand
                 }
 
                 $dependencyPackage = $this->gpm->findPackage($dependency);
-                $question = new ConfirmationQuestion("  |  '- Delete dependency <cyan>" . $dependency . "</cyan> too? [y|N] ", false);
-                $answer = $questionHelper->ask($this->input, $this->output, $question);
 
-                if ($answer) {
-                    $this->output->writeln("  |     '- <red>You decided to delete " . $dependency . ".</red>");
+                $dependency_exists = $this->packageExists($dependency, $dependencyPackage);
 
-                    $uninstall = $this->uninstallPackage($dependency, $dependencyPackage, true);
+                if ($dependency_exists == Installer::EXISTS) {
+                    $this->output->writeln("A dependency on <cyan>" . $dependencyPackage->name . "</cyan> [v" . $dependencyPackage->version . "] was found");
 
-                    if (!$uninstall) {
-                        $this->output->writeln("  '- <red>Uninstallation failed or aborted.</red>");
+                    $question = new ConfirmationQuestion("  |- Uninstall <cyan>" . $dependencyPackage->name . "</cyan>? [y|N] ", false);
+                    $answer = $this->all_yes ? true : $questionHelper->ask($this->input, $this->output, $question);
+
+                    if ($answer) {
+                        $uninstall = $this->uninstallPackage($dependency, $dependencyPackage, true);
+
+                        if (!$uninstall) {
+                            $this->output->writeln("  '- <red>Uninstallation failed or aborted.</red>");
+                        } else {
+                            $this->output->writeln("  '- <green>Success!</green>  ");
+
+                        }
                         $this->output->writeln('');
                     } else {
-                        $this->output->writeln("  '- <green>Success!</green>  ");
+                        $this->output->writeln("  '- <yellow>You decided not to uninstall " . $dependencyPackage->name . ".</yellow>");
                         $this->output->writeln('');
                     }
                 }
+
             }
         }
 
+
+        $locator = Grav::instance()['locator'];
+        $path = $locator->findResource($package->package_type . '://' . $slug);
+        Installer::uninstall($path);
+        $errorCode = Installer::lastErrorCode();
+
+        if ($errorCode && $errorCode !== Installer::IS_LINK && $errorCode !== Installer::EXISTS) {
+            $this->output->writeln("  |- Uninstalling " . $package->name . " package...  <red>error</red>                             ");
+            $this->output->writeln("  |  '- <yellow>" . Installer::lastErrorMsg()."</yellow>");
+
+            return false;
+        }
+
+        $message = Installer::getMessage();
+        if ($message) {
+            $this->output->writeln("  |- " . $message);
+        }
+
+        if (!$is_dependency && $this->dependencies) {
+            $this->output->writeln("Finishing up uninstalling <cyan>" . $package->name . "</cyan>");
+        }
+        $this->output->writeln("  |- Uninstalling " . $package->name . " package...  <green>ok</green>                             ");
+
+
+
         return true;
     }
 
@@ -225,17 +255,15 @@ class UninstallCommand extends ConsoleCommand
 
     private function checkDestination($slug, $package)
     {
-        $path = Grav::instance()['locator']->findResource($package->package_type . '://' . $slug);
         $questionHelper = $this->getHelper('question');
-        $skipPrompt = $this->input->getOption('all-yes');
 
-        Installer::isValidDestination($path);
+        $exists = $this->packageExists($slug, $package);
 
-        if (Installer::lastErrorCode() == Installer::IS_LINK) {
+        if ($exists == Installer::IS_LINK) {
             $this->output->write("\x0D");
             $this->output->writeln("  |- Checking destination...  <yellow>symbolic link</yellow>");
 
-            if ($skipPrompt) {
+            if ($this->all_yes) {
                 $this->output->writeln("  |     '- <yellow>Skipped automatically.</yellow>");
 
                 return false;
@@ -243,10 +271,10 @@ class UninstallCommand extends ConsoleCommand
 
             $question = new ConfirmationQuestion("  |  '- Destination has been detected as symlink, delete symbolic link first? [y|N] ",
                 false);
-            $answer = $questionHelper->ask($this->input, $this->output, $question);
+            $answer = $this->all_yes ? true : $questionHelper->ask($this->input, $this->output, $question);
 
             if (!$answer) {
-                $this->output->writeln("  |     '- <red>You decided to not delete the symlink automatically.</red>");
+                $this->output->writeln("  |     '- <red>You decided not to delete the symlink automatically.</red>");
 
                 return false;
             }
@@ -257,4 +285,18 @@ class UninstallCommand extends ConsoleCommand
 
         return true;
     }
+
+    /**
+     * Check if package exists
+     *
+     * @param $slug
+     * @param $package
+     * @return int
+     */
+    private function packageExists($slug, $package)
+    {
+        $path = Grav::instance()['locator']->findResource($package->package_type . '://' . $slug);
+        Installer::isValidDestination($path);
+        return Installer::lastErrorCode();
+    }
 }

system/src/Grav/Console/Gpm/UpdateCommand.php

@@ -47,6 +47,10 @@ class UpdateCommand extends ConsoleCommand
      */
     protected $gpm;
 
+    protected $all_yes;
+
+    protected $overwrite;
+
     /**
      *
      */
@@ -73,6 +77,12 @@ class UpdateCommand extends ConsoleCommand
                 InputOption::VALUE_NONE,
                 'Assumes yes (or best approach) instead of prompting'
             )
+            ->addOption(
+                'overwrite',
+                'o',
+                InputOption::VALUE_NONE,
+                'Option to overwrite packages if they already exist'
+            )
             ->addOption(
                 'plugins',
                 'p',
@@ -101,31 +111,40 @@ class UpdateCommand extends ConsoleCommand
     {
         $this->gpm = new GPM($this->input->getOption('force'));
 
+        $this->all_yes = $this->input->getOption('all-yes');
+        $this->overwrite = $this->input->getOption('overwrite');
+
         $this->displayGPMRelease();
 
         $this->destination = realpath($this->input->getOption('destination'));
-        $skip_prompt = $this->input->getOption('all-yes');
 
         if (!Installer::isGravInstance($this->destination)) {
             $this->output->writeln("<red>ERROR</red>: " . Installer::lastErrorMsg());
             exit;
         }
-        if ($this->input->getOption('plugins') === false and $this->input->getOption('themes') === false) {
-            $list_type_update = ['plugins' => true, 'themes' => true];
+        if ($this->input->getOption('plugins') === false && $this->input->getOption('themes') === false) {
+            $list_type = ['plugins' => true, 'themes' => true];
         } else {
-            $list_type_update['plugins'] = $this->input->getOption('plugins');
-            $list_type_update['themes'] = $this->input->getOption('themes');
+            $list_type['plugins'] = $this->input->getOption('plugins');
+            $list_type['themes'] = $this->input->getOption('themes');
+        }
+
+        if ($this->overwrite) {
+            $this->data = $this->gpm->getInstallable($list_type);
+            $description = " can be overwritten";
+        } else {
+            $this->data = $this->gpm->getUpdatable($list_type);
+            $description = " need updating";
         }
 
-        $this->data = $this->gpm->getUpdatable($list_type_update);
         $only_packages = array_map('strtolower', $this->input->getArgument('package'));
 
-        if (!$this->data['total']) {
+        if (!$this->overwrite && !$this->data['total']) {
             $this->output->writeln("Nothing to update.");
             exit;
         }
 
-        $this->output->write("Found <green>" . $this->gpm->countInstalled() . "</green> extensions installed of which <magenta>" . $this->data['total'] . "</magenta> need updating");
+        $this->output->write("Found <green>" . $this->gpm->countInstalled() . "</green> packages installed of which <magenta>" . $this->data['total'] . "</magenta>" . $description);
 
         $limit_to = $this->userInputPackages($only_packages);
 
@@ -145,19 +164,23 @@ class UpdateCommand extends ConsoleCommand
                     continue;
                 }
 
+                if (!$package->available) {
+                    $package->available = $package->version;
+                }
+
                 $this->output->writeln(
                 // index
                     str_pad($index++ + 1, 2, '0', STR_PAD_LEFT) . ". " .
                     // name
                     "<cyan>" . str_pad($package->name, 15) . "</cyan> " .
                     // version
-                    "[v<magenta>" . $package->version . "</magenta> ➜ v<green>" . $package->available . "</green>]"
+                    "[v<magenta>" . $package->version . "</magenta> -> v<green>" . $package->available . "</green>]"
                 );
                 $slugs[] = $slug;
             }
         }
 
-        if (!$skip_prompt) {
+        if (!$this->all_yes) {
             // prompt to continue
             $this->output->writeln("");
             $questionHelper = $this->getHelper('question');
@@ -165,7 +188,7 @@ class UpdateCommand extends ConsoleCommand
             $answer = $questionHelper->ask($this->input, $this->output, $question);
 
             if (!$answer) {
-                $this->output->writeln("Update aborted. Exiting...");
+                $this->output->writeln("<red>Update aborted. Exiting...</red>");
                 exit;
             }
         }
@@ -183,7 +206,7 @@ class UpdateCommand extends ConsoleCommand
         $command_exec = $install_command->run($args, $this->output);
 
         if ($command_exec != 0) {
-            $this->output->writeln("<red>Error:</red> An error occurred while trying to install the extensions");
+            $this->output->writeln("<red>Error:</red> An error occurred while trying to install the packages");
             exit;
         }
     }
@@ -204,7 +227,7 @@ class UpdateCommand extends ConsoleCommand
             foreach ($only_packages as $only_package) {
                 $find = $this->gpm->findPackage($only_package);
 
-                if (!$find || !$this->gpm->isUpdatable($find->slug)) {
+                if (!$find || (!$this->overwrite && !$this->gpm->isUpdatable($find->slug))) {
                     $name = isset($find->slug) ? $find->slug : $only_package;
                     $ignore[$name] = $name;
                 } else {

system/src/Grav/Console/TerminalObjects/Table.php

@@ -0,0 +1,29 @@
+<?php
+/**
+ * @package    Grav.Console
+ *
+ * @copyright  Copyright (C) 2014 - 2016 RocketTheme, LLC. All rights reserved.
+ * @license    MIT License; see LICENSE file for details.
+ */
+
+namespace Grav\Console\TerminalObjects;
+
+class Table extends \League\CLImate\TerminalObject\Basic\Table
+{
+    public function result()
+    {
+        $this->column_widths = $this->getColumnWidths();
+        $this->table_width   = $this->getWidth();
+        $this->border        = $this->getBorder();
+
+        $this->buildHeaderRow();
+
+        foreach ($this->data as $key => $columns) {
+            $this->rows[] = $this->buildRow($columns);
+        }
+
+        $this->rows[] = $this->border;
+
+        return $this->rows;
+    }
+}

tests/unit/Grav/Common/Helpers/ExcerptsTest.php

@@ -0,0 +1,85 @@
+<?php
+
+use Codeception\Util\Fixtures;
+use Grav\Common\Helpers\Excerpts;
+use Grav\Common\Grav;
+use Grav\Common\Uri;
+use Grav\Common\Config\Config;
+use Grav\Common\Page\Pages;
+use Grav\Common\Language\Language;
+
+/**
+ * Class ExcerptsTest
+ */
+class ExcerptsTest extends \Codeception\TestCase\Test
+{
+    /** @var Parsedown $parsedown */
+    protected $parsedown;
+
+    /** @var Grav $grav */
+    protected $grav;
+
+    /** @var Page $page */
+    protected $page;
+
+    /** @var Pages $pages */
+    protected $pages;
+
+    /** @var Config $config */
+    protected $config;
+
+    /** @var  Uri $uri */
+    protected $uri;
+
+    /** @var  Language $language */
+    protected $language;
+
+    protected $old_home;
+
+    protected function _before()
+    {
+        $grav = Fixtures::get('grav');
+        $this->grav = $grav();
+        $this->pages = $this->grav['pages'];
+        $this->config = $this->grav['config'];
+        $this->uri = $this->grav['uri'];
+        $this->language = $this->grav['language'];
+        $this->old_home = $this->config->get('system.home.alias');
+        $this->config->set('system.home.alias', '/item1');
+        $this->config->set('system.absolute_urls', false);
+        $this->config->set('system.languages.supported', []);
+
+        unset($this->grav['language']);
+        $this->grav['language'] = new Language($this->grav);
+
+        /** @var UniformResourceLocator $locator */
+        $locator = $this->grav['locator'];
+        $locator->addPath('page', '', 'tests/fake/nested-site/user/pages', false);
+        $this->pages->init();
+
+        $defaults = [
+            'extra'            => false,
+            'auto_line_breaks' => false,
+            'auto_url_links'   => false,
+            'escape_markup'    => false,
+            'special_chars'    => ['>' => 'gt', '<' => 'lt'],
+        ];
+        $this->page = $this->pages->dispatch('/item2/item2-2');
+        $this->uri->initializeWithURL('http://testing.dev/item2/item2-2')->init();
+    }
+
+    protected function _after()
+    {
+        $this->config->set('system.home.alias', $this->old_home);
+    }
+
+
+    public function testProcessImageHtml()
+    {
+        $this->assertRegexp('|<img alt="Sample Image" src="\/images\/.*-sample-image.jpe?g\" data-src="sample-image\.jpg\?cropZoom=300,300" \/>|',
+            Excerpts::processImageHtml('<img src="sample-image.jpg?cropZoom=300,300" alt="Sample Image" />', $this->page));
+        $this->assertRegexp('|<img alt="Sample Image" class="foo" src="\/images\/.*-sample-image.jpe?g\" data-src="sample-image\.jpg\?classes=foo" \/>|',
+            Excerpts::processImageHtml('<img src="sample-image.jpg?classes=foo" alt="Sample Image" />', $this->page));
+    }
+
+}

tests/unit/Grav/Common/Language/LanguageCodesTest.php

@@ -0,0 +1,24 @@
+<?php
+
+use Grav\Common\Language\LanguageCodes;
+
+
+/**
+ * Class ParsedownTest
+ */
+class LanguageCodesTest extends \Codeception\TestCase\Test
+{
+    public function testRtl()
+    {
+        $this->assertSame('ltr',
+            LanguageCodes::getOrientation('en'));
+        $this->assertSame('rtl',
+            LanguageCodes::getOrientation('ar'));
+        $this->assertSame('rtl',
+            LanguageCodes::getOrientation('he'));
+        $this->assertTrue(LanguageCodes::isRtl('ar'));
+        $this->assertFalse(LanguageCodes::isRtl('fr'));
+
+    }
+
+}

tests/unit/Grav/Common/Markdown/ParsedownTest.php

@@ -73,6 +73,16 @@ class ParsedownTest extends \Codeception\TestCase\Test
 
     public function testImages()
     {
+        $this->config->set('system.languages.supported', ['fr','en']);
+        unset($this->grav['language']);
+        $this->grav['language'] = new Language($this->grav);
+        $this->uri->initializeWithURL('http://testing.dev/fr/item2/item2-2')->init();
+
+        $this->assertSame('<p><img src="/tests/fake/nested-site/user/pages/02.item2/02.item2-2/sample-image.jpg" /></p>',
+            $this->parsedown->text('![](sample-image.jpg)'));
+        $this->assertRegexp('|<p><img src="\/images\/.*-cache-image.jpe?g\?foo=1" \/><\/p>|',
+            $this->parsedown->text('![](cache-image.jpg?cropResize=200,200&foo)'));
+
         $this->uri->initializeWithURL('http://testing.dev/item2/item2-2')->init();
 
         $this->assertSame('<p><img src="/tests/fake/nested-site/user/pages/02.item2/02.item2-2/sample-image.jpg" /></p>',
@@ -85,16 +95,11 @@ class ParsedownTest extends \Codeception\TestCase\Test
             $this->parsedown->text('![](missing-image.jpg)'));
         $this->assertSame('<p><img src="/home-missing-image.jpg" alt="" /></p>',
             $this->parsedown->text('![](/home-missing-image.jpg)'));
+        $this->assertSame('<p><img src="/home-missing-image.jpg" alt="" /></p>',
+            $this->parsedown->text('![](/home-missing-image.jpg)'));
+        $this->assertSame('<p><img src="https://getgrav-grav.netdna-ssl.com/user/pages/media/grav-logo.svg" alt="" /></p>',
+            $this->parsedown->text('![](https://getgrav-grav.netdna-ssl.com/user/pages/media/grav-logo.svg)'));
 
-        $this->config->set('system.languages.supported', ['fr','en']);
-        unset($this->grav['language']);
-        $this->grav['language'] = new Language($this->grav);
-        $this->uri->initializeWithURL('http://testing.dev/fr/item2/item2-2')->init();
-
-        $this->assertSame('<p><img src="/tests/fake/nested-site/user/pages/02.item2/02.item2-2/sample-image.jpg" /></p>',
-            $this->parsedown->text('![](sample-image.jpg)'));
-        $this->assertRegexp('|<p><img src="\/images\/.*-cache-image.jpe?g\?foo=1" \/><\/p>|',
-            $this->parsedown->text('![](cache-image.jpg?cropResize=200,200&foo)'));
    }
 
     public function testImagesSubDir()

tests/unit/Grav/Common/Twig/TwigExtensionTest.php

@@ -30,7 +30,7 @@ class TwigExtensionTest extends \Codeception\TestCase\Test
         $this->assertSame('camel_cased',                $this->twig_ext->inflectorFilter('underscor',    'CamelCased'));
         $this->assertSame('something-text',             $this->twig_ext->inflectorFilter('hyphen',       'Something Text'));
         $this->assertSame('Something text to read',     $this->twig_ext->inflectorFilter('human',        'something_text_to_read'));
-        $this->assertSame(5,                            $this->twig_ext->inflectorFilter('month',        181));
+        $this->assertSame(5,                            $this->twig_ext->inflectorFilter('month',        175));
         $this->assertSame('10th',                       $this->twig_ext->inflectorFilter('ordinal',      10));
     }
 
@@ -70,12 +70,6 @@ class TwigExtensionTest extends \Codeception\TestCase\Test
         }
     }
 
-    public function testSafeEmailFilter()
-    {
-        $this->assertSame('devs@getgrav.org',   $this->twig_ext->safeEmailFilter('devs@getgrav.org'));
-        $this->assertSame('someone@example.com',   $this->twig_ext->safeEmailFilter('someone@example.com'));
-    }
-
     public function testRandomizeFilter()
     {
         $array = [1,2,3,4,5];

tests/unit/Grav/Common/UtilsTest.php

@@ -123,16 +123,22 @@ class UtilsTest extends \Codeception\TestCase\Test
 
     public function testTruncateHtml()
     {
-        $this->assertEquals('<p>T…</p>', Utils::truncateHtml('<p>This is a string to truncate</p>', 1));
-        $this->assertEquals('<p>This…</p>', Utils::truncateHtml('<p>This is a string to truncate</p>', 4));
-        $this->assertEquals('', Utils::truncateHtml('<input type="file" id="file" multiple />', 6, true));
-
+        $this->assertEquals('<p>T...</p>', Utils::truncateHtml('<p>This is a string to truncate</p>', 1));
+        $this->assertEquals('<p>This...</p>', Utils::truncateHtml('<p>This is a string to truncate</p>', 4));
+        $this->assertEquals('<p>This is a...</p>', Utils::truncateHtml('<p>This is a string to truncate</p>', 10));
+        $this->assertEquals('<p>This is a string to truncate</p>', Utils::truncateHtml('<p>This is a string to truncate</p>', 100));
+        $this->assertEquals('<input type="file" id="file" multiple>', Utils::truncateHtml('<input type="file" id="file" multiple />', 6));
+        $this->assertEquals('<ol><li>item 1 <i>so...</i></li></ol>', Utils::truncateHtml('<ol><li>item 1 <i>something</i></li><li>item 2 <strong>bold</strong></li></ol>', 10));
     }
 
     public function testSafeTruncateHtml()
     {
-        $this->assertEquals('<p>This…</p>', Utils::safeTruncateHtml('<p>This is a string to truncate</p>', 1));
-        $this->assertEquals('<p>This…</p>', Utils::safeTruncateHtml('<p>This is a string to truncate</p>', 4));
+        $this->assertEquals('<p>This...</p>', Utils::safeTruncateHtml('<p>This is a string to truncate</p>', 1));
+        $this->assertEquals('<p>This is...</p>', Utils::safeTruncateHtml('<p>This is a string to truncate</p>', 2));
+        $this->assertEquals('<p>This is a string to...</p>', Utils::safeTruncateHtml('<p>This is a string to truncate</p>', 5));
+        $this->assertEquals('<p>This is a string to truncate</p>', Utils::safeTruncateHtml('<p>This is a string to truncate</p>', 20));
+        $this->assertEquals('<input type="file" id="file" multiple>', Utils::safeTruncateHtml('<input type="file" id="file" multiple />', 6));
+        $this->assertEquals('<ol><li>item 1 <i>something</i></li><li>item 2...</li></ol>', Utils::safeTruncateHtml('<ol><li>item 1 <i>something</i></li><li>item 2 <strong>bold</strong></li></ol>', 5));
     }
 
     public function testGenerateRandomString()
@@ -146,13 +152,36 @@ class UtilsTest extends \Codeception\TestCase\Test
 
     }
 
-    public function testGetMimeType()
+    public function testGetMimeByExtension()
     {
-        $this->assertEquals('application/octet-stream', Utils::getMimeType(''));
-        $this->assertEquals('image/jpeg', Utils::getMimeType('jpg'));
-        $this->assertEquals('image/png', Utils::getMimeType('png'));
-        $this->assertEquals('text/plain', Utils::getMimeType('txt'));
-        $this->assertEquals('application/msword', Utils::getMimeType('doc'));
+        $this->assertEquals('application/octet-stream', Utils::getMimeByExtension(''));
+        $this->assertEquals('text/html', Utils::getMimeByExtension('html'));
+        $this->assertEquals('application/json', Utils::getMimeByExtension('json'));
+        $this->assertEquals('application/atom+xml', Utils::getMimeByExtension('atom'));
+        $this->assertEquals('application/rss+xml', Utils::getMimeByExtension('rss'));
+        $this->assertEquals('image/jpeg', Utils::getMimeByExtension('jpg'));
+        $this->assertEquals('image/png', Utils::getMimeByExtension('png'));
+        $this->assertEquals('text/plain', Utils::getMimeByExtension('txt'));
+        $this->assertEquals('application/msword', Utils::getMimeByExtension('doc'));
+        $this->assertEquals('application/octet-stream', Utils::getMimeByExtension('foo'));
+        $this->assertEquals('foo/bar', Utils::getMimeByExtension('foo', 'foo/bar'));
+        $this->assertEquals('text/html', Utils::getMimeByExtension('foo', 'text/html'));
+    }
+
+    public function testGetExtensionByMime()
+    {
+        $this->assertEquals('html', Utils::getExtensionByMime('*/*'));
+        $this->assertEquals('html', Utils::getExtensionByMime('text/*'));
+        $this->assertEquals('html', Utils::getExtensionByMime('text/html'));
+        $this->assertEquals('json', Utils::getExtensionByMime('application/json'));
+        $this->assertEquals('atom', Utils::getExtensionByMime('application/atom+xml'));
+        $this->assertEquals('rss', Utils::getExtensionByMime('application/rss+xml'));
+        $this->assertEquals('jpg', Utils::getExtensionByMime('image/jpeg'));
+        $this->assertEquals('png', Utils::getExtensionByMime('image/png'));
+        $this->assertEquals('txt', Utils::getExtensionByMime('text/plain'));
+        $this->assertEquals('doc', Utils::getExtensionByMime('application/msword'));
+        $this->assertEquals('html', Utils::getExtensionByMime('foo/bar'));
+        $this->assertEquals('baz', Utils::getExtensionByMime('foo/bar', 'baz'));
     }
 
     public function testNormalizePath()

user/config/system.yaml

@@ -5,7 +5,8 @@ home:
 
 pages:
   theme: antimatter
-  markdown_extra: false
+  markdown:
+    extra: false
   process:
     markdown: true
     twig: false
@@ -39,3 +40,5 @@ debugger:
   twig: true
   shutdown:
     close_connection: true
+gpm:
+  verify_peer: true