cee7479cf4
* Configurations update for rails 4.0 * ActiveSupport::Testing::Performance extracted to a gem * ruby prof required as a dependency * disable rubocop on bin folder * http patch * Comment out to allow testing * no longer supports plugin loading * lock to sprockets 2.12 * Active record patches * remove deprecated test syntax * fix failing tests * change new super class * replace right_aws with right_aws_api right_aws is no longer maintained, was throwing an error * lock to rails 4.2.11.1 * change rails version in install script * remove deprecation warnings * Change test lib to minitest, add minitest reporters * make app work * active record find patches * root_in_json include defaulted to false * confirm option removed in link helper * cookies serializer changed to hybrid * Change render :text to :plain render :text will be deprecated and poses a security risk * console for dev web * Check and fix interface functionality * fix export functionality * add protected attributes for the warpable model * fix image upload * Fix comments and images failing tests * include mass assignment security in annotations * render html for update images * clear mail array before every test * Fix codeclimate issues * skip failing test The test is failing because of different names in model(warpable) and controller(images) skipping this for now until we decide if we want to standardize the names * replace unprotected redirects * Fix codeclimate issues * Autofixing rubocop offenses and Problematic test for #578 (#667) * Autofixing rubocop offenses * Adding Rubocop to Travis pipelines and development branch * Fixing maps controller test * Fixing remaining offenses * change post test to create since action new is a get action * remove unprotected redirects * Fix codeclimate issues * Add yarn * Remove error log * Fix oauth icons * Modify package.json * Add yarn install to start.sh * Add leaflet google * Remove leaflet-google from package json * remove passenger error logs * Fix install script * Remove flag * Fix gemfile.lock * Fix gemfile.lock * Fix login * Upgrade Gemfile to Rails 5.0 * require rake' * Change config files * Add application record * Bump mysql * Local builds for Travis runners (#672) * Using local mysql for travis * Fix codeclimate issues * Refactoring yamls * Autofixing rubocop offenses and Problematic test for #578 (#667) * Autofixing rubocop offenses * Adding Rubocop to Travis pipelines and development branch * Fixing maps controller test * Fixing remaining offenses * Fixing unit tests * Adding docker build to travis pipelines * Adding docker build to travis pipelines * Staging builds in travis * fix travis.yml * Upgrading sintax of assets and using required gems * Using updated version of GDAL and installing required dependencies * Enabling cache in between builds * Test yarn for travis * Conditionalize rake db:setup for travis * Add semicolon * Remove comment * modify database.yml * Migrate seperately * Run create only for production * Locking newer Rails v * Running update task * Adding missing bootsnap gem * Fixing missing database * Adding missing listen gem * Fixing schema example version * Fixing migration versions * Updating dependencies for Rails 5 * Adding ApplicationMailer abstraction * Adding required initializers * Prefer require_relative instead of full path * Making associations not required by default * Regenerating schema file * Hotfixing MassAssigment * Upgrade web-console, remove mysql adapter override * Remove attr_accessible * Add rails-controller-testing gem * Follow new syntax for tests * Remove extra web-console from gemfile * Regenerating lock file * Fixing rubocop offenses and bundler version * Using correct Paperclip class as in https://github.com/rails/rails/issues/26404#issuecomment-502129936 * Fix images functional tests * images controller test typecast to string * Fix rails logger * Fix map tests * Bumping rubocop version * Autofixing rubocop offenses * Including performance cop to rubocop * Refactoring deprecated routing and secret_token * Fixing routes for feed controller * Using correct routes for RSS builder * Fixing missing routes * Fixing travis bundler and yarn cache * Fixing bundle path * Splitting bundle and yarn verifications * Autofixing rubocop offenses * Fixing prod host for travis, private class usage and rubocop offenses * Upgrade to Rails 5.2 (#685) * Upgrade Gemfile to Rails 5.0 * require rake' * Change config files * Add application record * Bump mysql * Locking newer Rails v * Running update task * Adding missing bootsnap gem * Fixing missing database * Adding missing listen gem * Fixing schema example version * Fixing migration versions * Updating dependencies for Rails 5 * Adding ApplicationMailer abstraction * Adding required initializers * Prefer require_relative instead of full path * Making associations not required by default * Regenerating schema file * Hotfixing MassAssigment * Upgrade web-console, remove mysql adapter override * Remove attr_accessible * Add rails-controller-testing gem * Follow new syntax for tests * Remove extra web-console from gemfile * Regenerating lock file * Fixing rubocop offenses and bundler version * Using correct Paperclip class as in https://github.com/rails/rails/issues/26404#issuecomment-502129936 * Fix images functional tests * images controller test typecast to string * Fix rails logger * Fix map tests * Bumping rubocop version * Autofixing rubocop offenses * Including performance cop to rubocop * Refactoring deprecated routing and secret_token * Fixing routes for feed controller * Using correct routes for RSS builder * Fixing missing routes * Fixing travis bundler and yarn cache * Fixing bundle path * Splitting bundle and yarn verifications * Fixing prod host for travis, private class usage and rubocop offenses * Enforcing params usage on get method * Using correct folder names to Rails >5 conventions * Enforcing params wrapping and adding missing front_ui route * Precompiling assets before serving * Improving Jenkins startup script * WIP action cable setup * basic action cable setup complete * minor change * minor changes * few changes * Using supported docker yaml version by Jenkins * Adding task to check database existance * Improving start script * Improving Makefile's recipes and target * Adding task to check database existance * Improving start script * Improving Makefile's recipes and target * Improving Makefile's recipes and target * Patching https://github.com/publiclab/mapknitter/pull/803 * Improving Jenkins setup * initial working functionality complete * Fix map loading * h * Fixing Leaflet-Environmental-Layers map loading * h * leaflet * fix * change * updates * stop precompiling assets * precompile * Using correct Yarn, NPM and Node version, avoiding mismatch * Removing unwanted tags.js invocation * Improving Makefile recipe * Using node_modules/ as dependencies folder, since https://github.com/sass/node-sass/issues/2050#issuecomment-317233552 * Upgrading Yarn dependencies * Removing duplicate rubocop directive * Removing test/ from codeclimate checks * Removing fixed FIXME comments * Removing fixed FIXME comments * Updating docs in README * Refactoring code * Bumping recaptcha and include methods * Adding Foreman gem * Scheduling Puma and Passenger servers * WIP action cable setup * basic action cable setup complete * minor change * minor changes * few changes * initial working functionality complete * Refactoring code * Adding Foreman gem * Scheduling Puma and Passenger servers * few minor fix * added a few tests * Refactoring connection module * Fixing migration version * Using strong params in requests * Using strong params in requests * Use Rack::Test::UploadedFile instead of ActionDispatch::Http::UploadedFile * added documentation * added more docs * added tests * Fix minor asset issue * Remove manual asset references and add them to application js * Fix asset ordering in application.js * Configure System tests (#936) * Add new system tests and fix minor asset loading * modify test * Add chromedriver to travis * Add sudo * Add dependencies to dockerfile * Properly installing chrome and chromedriver * Using puma as dependency and correct image controller * added a few tests * a few changes * remove unnecessary render * few test fixes * action cable setup (#805) * WIP action cable setup * basic action cable setup complete * minor change * minor changes * few changes * initial working functionality complete * Refactoring code * Adding Foreman gem * Scheduling Puma and Passenger servers * WIP action cable setup * basic action cable setup complete * minor change * minor changes * few changes * initial working functionality complete * Refactoring code * Adding Foreman gem * Scheduling Puma and Passenger servers * few minor fix * added a few tests * Refactoring connection module * Using strong params in requests * added documentation * added more docs * added tests * Using puma as dependency and correct image controller * added a few tests * a few changes * remove unnecessary render * few test fixes * Fixing CodeClimate issues * Synch editing add ons (#957) * few bug fixes * separate editing channels for different maps * test fixes * rubocop fixes * Undoing unwanted pattern set by Rubocop
MapKnitter
Use Public Lab's open source MapKnitter to upload your own aerial photographs (for example those from balloon or kite mapping: http://publiclab.org/balloon-mapping) and combine them into:
- Web "slippy maps" like Google Maps
- GeoTiff
- TMS
- High resolution JPEG
Table of Contents
- Architecture
- Installation
- Logging in when running locally
- Bugs and support
- Developers
- Staging infrastructure and testing
- License
Architecture
MapKnitter is broken into three major components:
- Map user interface
- Application
- Exporting system
Component 1 has been broken out into a new Leaflet plugin, Leaflet.DistortableImage, which allows for client-side, CSS3-based distortion of images over a Leaflet base map
Component 2 is a Ruby on Rails application which is the core of what you've looked at. It stores images, image corner locations, annotations, map details, and user accounts.
Component 3 is a set of calls to GDAL (Geospatial Data Abstraction Library) and ImageMagick, which perform the distortions, geolocations, and produce export products like GeoTiff, TMS, jpg, etc. These are baked into the Warpable and Map models, as well as the Export controller, and could use some consolidation.
Component 3 is soon to be replaced with an external exporter service built in a small Sinatra app called mapknitter-exporter-sinatra using the mapknitter-exporter gem.
Another moving part is the new-ish Annotations 2.0 which uses Leaflet.Illustrate to provide rich annotation on top of maps.
Installation
Please consider which installation method you prefer. Cloud Installation requires fewer steps and is platform agnostic, but you may value working from your terminal, for familiarity, more.
Standard Installation
Prerequisites
Make you have the below 3 prerequisites installed before moving forward with the Installation Steps.
Instructions are for an Ubuntu/Debian system. Varies slightly for mac/fedora/etc.
MySQL
- MacOS and Linux users, please reference MYSQL.md instead.
Install a database, if necessary. sqlite does not seem to work due to some table constraints:
$ sudo apt-get install mysql-server
Application-specific dependencies:
$ sudo apt-get install bundler libmysqlclient-dev imagemagick ruby-rmagick libfreeimage3 libfreeimage-dev ruby-dev libmagickcore-dev libmagickwand-dev
(optional) For exporting, you'll need GDAL >=1.7.x (gdal.org), as well as curl and zip-- but these are not needed for much of development, unless you're working on the exporting features.
$ sudo apt-get install gdal-bin python-gdal curl libcurl4-openssl-dev libssl-dev zip
==================
Ruby version manager: RVM / Rbenv
This is for RVM, but the alternative, Rbenv, also works (instructions not listed here). Don't install RVM if you already have Rbenv!
Install RVM for Ruby management (http://rvm.io)
$ curl -L https://get.rvm.io | bash -s stable
Note: At this point during the process, you may want to log out and log back in, or open a new terminal window; RVM will then properly load in your environment.
Ubuntu users: You may need to enable Run command as a login shell in Ubuntu's Terminal, under Profile Preferences > Title and Command. Then close the terminal and reopen it.
Then, use RVM to install version 2.4.6 of Ruby:
$ rvm install 2.4.6
==================
Package manager: Npm and Yarn
You'll also need yarn which is available through NPM. To install npm, you can run:
$ sudo apt-get install npm
However, on Ubuntu, you may need to also install the nodejs-legacy package, as due to a naming collision, some versions of Ubuntu already have an unrelated package called node. To do this, run:
$ sudo apt-get install nodejs-legacy
Once NPM is installed, you should be able to run:
$ sudo npm install -g yarn
==================
Installation Steps
You'll need Ruby v2.4.6 (use your local ruby version management system - RVM / rbenv / etc. - to install and set locally)
- Download a copy of the source with
git clone https://github.com/publiclab/mapknitter.git - Install gems with
bundle installfrom the rails root folder. You may need to runbundle updateif you have older gems in your environment. - Copy and configure config/database.yml from config/database.yml.example, using a new empty database you've created
- Copy and configure config/config.yml from config/config.yml.example (for now, this is only for the Google Maps API Key, which is optional, and a path for logging in when running locally, also optional)
- Initialize database with
bundle exec rails db:setup - Enter ReCaptcha public and private keys in config/initializers/recaptcha.rb, copied from recaptcha.rb.example. To get keys, visit https://www.google.com/recaptcha/admin/create
- Install static assets (like external javascript libraries, fonts) with
yarn install - Start rails with
bundle exec passenger startfrom the Rails root and open http://localhost:3000 in a web browser. (For some, justpassenger startwill work; addingbundle execensures you're using the version of passenger you just installed with Bundler.)
==================
Installation video
For a run-through of the Prerequisites and Installation steps listed below, you can watch the install video at:
http://youtu.be/iGYGpS8rZMY (may be slightly out of date, but gives an overview)
Cloud Installation
We provide an install script for Codenvy's cloud service, which provides a free developer workspace server that allows anyone to contribute to a project without installing software: https://Codenvy.io.
To use it:
- Create a personal account.
- Click
Create new workspace. - Select a new workspace with a
Railsstack. - Under the
Projectssection, add the URL of your forked version of mapknitter (https://github.com/USERNAME/mapknitter.git). - Hit create.
- It will open in the projects explorer - use the
bashconsole at the bottom of the screen tocdinto this project's directory. - Run the installation script. The initial installation may take a bit.
$ source install_cloud.sh
- When you see it's complete, run the server:
$ rails server -b 0.0.0.0
- Hit the Play button located in the top menu bar.
- Open the Codenvy URL provided in the console to see MapKnitter booted up. Great work!
Logging in when running locally
Because MapKnitter uses a remote OpenID login system that depends on PublicLab.org, it can be hard to log in when running it locally. To get around this, we've created a local login route that requires no password:
You can log in locally at the path http://localhost:3000/local/USERNAME where USERNAME is any username.
For this to work:
-
You will need to have copied and configured config/config.yml from config/config.yml.example
-
The user has to be an existing record. For your convenience, we have added two user accounts in seeds.rb to make their corresponding paths available in development after installation:
# basic account path - http://localhost:3000/local/harry
# created from:
User.create({login: 'harry', name: 'harry potter', email: 'potter@hogwarts.com'})
# admin account path - http://localhost:3000/local/albus
# created from:
u_admin = User.create({login: 'albus', name: 'albus dumbledore', email: 'dumbledore@hogwarts.com'})
u_admin.role = 'admin'
Running tests
When you try to run tests in MapKnitter, you can run the default Rake tasks, such as:
rails test:unit rails test:controllers rails test:integration
or simply:
rails test
Running tests of a specific file:
rails test test/unit/some_file.rb
Running a single test from the test suite:
rails test test/functional/some_file.rb:[line number of the test]
Bugs and support
To report bugs and request features, please use the GitHub issue tracker provided at https://github.com/publiclab/mapknitter/issues
For additional support, join the Public Lab website and mailing list at http://publiclab.org/lists or for urgent requests, email web@publiclab.org
For questions related to the use of this software and balloon or kite mapping, the same page links to the "grassrootsmapping" discussion group.
Developers
Help improve Public Lab software!
- Join the 'plots-dev@googlegroups.com' discussion list to get involved
- Look for open issues at https://github.com/publiclab/mapknitter/issues
- Review contributor guidelines at http://publiclab.org/wiki/contributing-to-public-lab-software
- Some devs hang out in http://publiclab.org/chat (irc webchat)
- Find lots of info on contributing at http://publiclab.org/wiki/developers
- Join our gitter chat at https://gitter.im/publiclab/publiclab
Staging infrastructure and testing
In addition automatic testing with Travis CI - we have a branch (unstable) is set to auto-build and deploy to a staging instance. This instance includes a copy of the production database and is intended for experimenting or debugging purposes with a production-like environment. We also have a stable build at http://mapknitter-stable.laboratoriopublico.org/ which builds off of our main branch. Any commits or PRs merged to the main branch will trigger the stable server to rebuild; you can monitor progress at https://jenkins.laboratoriopublico.org/
License
MapKnitter is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
MapKnitter is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with MapKnitter. If not, see http://www.gnu.org/licenses/.