dominik created page: Setup Metager authored by Dominik Hebeler's avatar Dominik Hebeler
This installs and setup MetaGer on a fresh debian installation.
Apache will be used as a webserver.
Redis will be used as Queue for the fetch jobs.
Supervisor will be used to keep the workers running that will fetch results from the search engines.
You can adapt those configurations to your needs.
This manual is tested on the latest stable debian version (stretch) but should also work on other distributions.
# Install Dependencies
1. `sudo apt -y update`
2. `sudo apt install -y git curl unzip build-essential libpng-dev supervisor apache2 redis-server php php-common php-curl php-mbstring php-mysql php-sqlite3 php-xml php-zip php-redis php-gd libapache2-mod-php`
=> build-essential will only be needed if npm has to build some packages from source
# Install the latest Composer Version
Steps from https://getcomposer.org/
1. `php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"`
2. `php composer-setup.php`
3. `php -r "unlink('composer-setup.php');"`
4. `sudo mv composer.phar /usr/bin/composer`
# Install the latest nodejs
Steps from https://github.com/nodesource/distributions/blob/master/README.md#debinstall
1. `curl -sL https://deb.nodesource.com/setup_11.x | bash -` (as root)
2. `sudo apt install -y nodejs`
# Download and install dependencies of MetaGer
1. `git clone https://gitlab.metager3.de/open-source/MetaGer.git`
2. `cd MetaGer`
3. `composer install`
4. `npm install`
5. `cp .env.example .env`
6. Edit `.env` and set `QUEUE_DRIVER=redis` (If you need to change settings for database and redis this would be the place, too)
7. `php artisan key:generate`
8. Set the correct Directory Permissions (web-server should be able to write to two dirs)
* `sudo chown -R :www-data storage && sudo chown -R :www-data bootstrap/cache`
* `sudo chmod -R g+w storage && sudo chmod -R g+w bootstrap/cache`
9. `npm run prod`
10. `cp config/sumas.xml.example config/sumas.xml` (Enables some example search engines)
=> We will refer to the path where you cloned MetaGer to as $metager_path (including the MetaGer folder)
# Make sure redis is running
1. `sudo service redis restart`
# Setup and configure supervisor
1. `cd /etc/supervisor/conf.d/`
2. Create the File laravel-worker.conf and put the following content
```
[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php $metager_path/artisan queue:work --sleep=1 --tries=1 --timeout=295 --daemon
autostart=true
autorestart=true
user=$username
numprocs=200
redirect_stderr=true
stdout_logfile=$metager_path/storage/logs/worker.log
```
=> Replace the $metager_path and set the user to the username you cloned MetaGer with
3. Restart/start the supervisor daemon `sudo service supervisor start` or `sudo service supervisor restart`
=> If it worked correctly you will see 200 php processes with `pgrep php`. You can change the number in the config to whatever you find reasonable. It defines how many requests can be made to your search engines in parallel.
# Setup and configure Apache
1. `cd /etc/apache2/sites-available/`
2. Edit 000-default.conf and put the following content
```
<VirtualHost *:80>
# If you have a DNS-Entry pointing to your machine you can comment out the following line and set the dns name there
#ServerName www.example.com
ServerAdmin webmaster@localhost
# Replace $metager_path with the actual path
DocumentRoot $metager_path/public
<Directory $metager_path/public>
Allow Override all
Require all granted
</Directory>
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>
```
3. `sudo a2enmod rewrite`
4. restart/start the apache server `sudo service apache2 restart`
If you want to modify/add searchengines to your MetaGer instance you will have to modify the search engine definition in $metager_path/config/sumas.xml and you will have to add a parser class for that search engine to $metager_path/app/Models/parserSkripte/.
With the information from the `sumas.xml` MetaGer will make a request to all search engines with `type=web`. The result will be given to above mentioned parser class where you have to write the code that transforms the result from the search engine API to a php object that MetaGer can use later.