|
|
|
This installs and setup MetaGer on a fresh debian installation.
|
|
|
|
Apache will be used as a webserver.
|
|
|
|
Redis will be used as Queue for the fetch jobs.
|
|
|
|
Supervisor will be used to keep the workers running that will fetch results from the search engines.
|
|
|
|
You can adapt those configurations to your needs.
|
|
|
|
This manual is tested on the latest stable debian version (stretch) but should also work on other distributions.
|
|
|
|
|
|
|
|
# Install Dependencies
|
|
|
|
1. `sudo apt -y update`
|
|
|
|
2. `sudo apt install -y git curl unzip build-essential libpng-dev supervisor apache2 redis-server php php-common php-curl php-mbstring php-mysql php-sqlite3 php-xml php-zip php-redis php-gd libapache2-mod-php`
|
|
|
|
|
|
|
|
=> build-essential will only be needed if npm has to build some packages from source
|
|
|
|
|
|
|
|
# Install the latest Composer Version
|
|
|
|
Steps from https://getcomposer.org/
|
|
|
|
1. `php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"`
|
|
|
|
2. `php composer-setup.php`
|
|
|
|
3. `php -r "unlink('composer-setup.php');"`
|
|
|
|
4. `sudo mv composer.phar /usr/bin/composer`
|
|
|
|
|
|
|
|
# Install the latest nodejs
|
|
|
|
Steps from https://github.com/nodesource/distributions/blob/master/README.md#debinstall
|
|
|
|
1. `curl -sL https://deb.nodesource.com/setup_11.x | bash -` (as root)
|
|
|
|
2. `sudo apt install -y nodejs`
|
|
|
|
|
|
|
|
# Download and install dependencies of MetaGer
|
|
|
|
1. `git clone https://gitlab.metager3.de/open-source/MetaGer.git`
|
|
|
|
2. `cd MetaGer`
|
|
|
|
3. `composer install`
|
|
|
|
4. `npm install`
|
|
|
|
5. `cp .env.example .env`
|
|
|
|
6. Edit `.env` and set `QUEUE_DRIVER=redis` (If you need to change settings for database and redis this would be the place, too)
|
|
|
|
7. `php artisan key:generate`
|
|
|
|
8. Set the correct Directory Permissions (web-server should be able to write to two dirs)
|
|
|
|
* `sudo chown -R :www-data storage && sudo chown -R :www-data bootstrap/cache`
|
|
|
|
* `sudo chmod -R g+w storage && sudo chmod -R g+w bootstrap/cache`
|
|
|
|
9. `npm run prod`
|
|
|
|
10. `cp config/sumas.xml.example config/sumas.xml` (Enables some example search engines)
|
|
|
|
|
|
|
|
=> We will refer to the path where you cloned MetaGer to as $metager_path (including the MetaGer folder)
|
|
|
|
|
|
|
|
# Make sure redis is running
|
|
|
|
1. `sudo service redis restart`
|
|
|
|
|
|
|
|
# Setup and configure supervisor
|
|
|
|
1. `cd /etc/supervisor/conf.d/`
|
|
|
|
2. Create the File laravel-worker.conf and put the following content
|
|
|
|
```
|
|
|
|
[program:laravel-worker]
|
|
|
|
process_name=%(program_name)s_%(process_num)02d
|
|
|
|
command=php $metager_path/artisan queue:work --sleep=1 --tries=1 --timeout=295 --daemon
|
|
|
|
autostart=true
|
|
|
|
autorestart=true
|
|
|
|
user=$username
|
|
|
|
numprocs=200
|
|
|
|
redirect_stderr=true
|
|
|
|
stdout_logfile=$metager_path/storage/logs/worker.log
|
|
|
|
```
|
|
|
|
=> Replace the $metager_path and set the user to the username you cloned MetaGer with
|
|
|
|
3. Restart/start the supervisor daemon `sudo service supervisor start` or `sudo service supervisor restart`
|
|
|
|
=> If it worked correctly you will see 200 php processes with `pgrep php`. You can change the number in the config to whatever you find reasonable. It defines how many requests can be made to your search engines in parallel.
|
|
|
|
|
|
|
|
# Setup and configure Apache
|
|
|
|
1. `cd /etc/apache2/sites-available/`
|
|
|
|
2. Edit 000-default.conf and put the following content
|
|
|
|
```
|
|
|
|
<VirtualHost *:80>
|
|
|
|
# If you have a DNS-Entry pointing to your machine you can comment out the following line and set the dns name there
|
|
|
|
#ServerName www.example.com
|
|
|
|
|
|
|
|
ServerAdmin webmaster@localhost
|
|
|
|
|
|
|
|
# Replace $metager_path with the actual path
|
|
|
|
DocumentRoot $metager_path/public
|
|
|
|
<Directory $metager_path/public>
|
|
|
|
Allow Override all
|
|
|
|
Require all granted
|
|
|
|
</Directory>
|
|
|
|
|
|
|
|
ErrorLog ${APACHE_LOG_DIR}/error.log
|
|
|
|
CustomLog ${APACHE_LOG_DIR}/access.log combined
|
|
|
|
|
|
|
|
</VirtualHost>
|
|
|
|
```
|
|
|
|
3. `sudo a2enmod rewrite`
|
|
|
|
4. restart/start the apache server `sudo service apache2 restart`
|
|
|
|
|
|
|
|
If you want to modify/add searchengines to your MetaGer instance you will have to modify the search engine definition in $metager_path/config/sumas.xml and you will have to add a parser class for that search engine to $metager_path/app/Models/parserSkripte/.
|
|
|
|
With the information from the `sumas.xml` MetaGer will make a request to all search engines with `type=web`. The result will be given to above mentioned parser class where you have to write the code that transforms the result from the search engine API to a php object that MetaGer can use later.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
