Skip to content

GitLab

  • Menu
Projects Groups Snippets
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • MetaGer MetaGer
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 63
    • Issues 63
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 10
    • Merge requests 10
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • Value stream
    • Repository
  • Wiki
    • Wiki
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
Collapse sidebar
  • open-source
  • MetaGerMetaGer
  • Wiki
  • Setup Metager

Last edited by Dominik Hebeler Nov 14, 2018
Page history

Setup Metager

This installs and setup MetaGer on a fresh debian installation.
Apache will be used as a webserver.
Redis will be used as Queue for the fetch jobs.
Supervisor will be used to keep the workers running that will fetch results from the search engines.
You can adapt those configurations to your needs.
This manual is tested on the latest stable debian version (stretch) but should also work on other distributions.

Install Dependencies

  1. sudo apt -y update
  2. sudo apt install -y git curl unzip build-essential libpng-dev supervisor apache2 redis-server php php-common php-curl php-mbstring php-mysql php-sqlite3 php-xml php-zip php-redis php-gd libapache2-mod-php

=> build-essential will only be needed if npm has to build some packages from source

Install the latest Composer Version

Steps from https://getcomposer.org/

  1. php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
  2. php composer-setup.php
  3. php -r "unlink('composer-setup.php');"
  4. sudo mv composer.phar /usr/bin/composer

Install the latest nodejs

Steps from https://github.com/nodesource/distributions/blob/master/README.md#debinstall

  1. curl -sL https://deb.nodesource.com/setup_11.x | bash - (as root)
  2. sudo apt install -y nodejs

Download and install dependencies of MetaGer

  1. git clone https://gitlab.metager3.de/open-source/MetaGer.git
  2. cd MetaGer
  3. composer install
  4. npm install
  5. cp .env.example .env
  6. Edit .env and set QUEUE_DRIVER=redis (If you need to change settings for database and redis this would be the place, too)
  7. php artisan key:generate
  8. Set the correct Directory Permissions (web-server should be able to write to two dirs)
    • sudo chown -R :www-data storage && sudo chown -R :www-data bootstrap/cache
    • sudo chmod -R g+w storage && sudo chmod -R g+w bootstrap/cache
  9. npm run prod
  10. cp config/sumas.xml.example config/sumas.xml (Enables some example search engines)

=> We will refer to the path where you cloned MetaGer to as $metager_path (including the MetaGer folder)

Make sure redis is running

  1. sudo service redis restart

Setup and configure supervisor

  1. cd /etc/supervisor/conf.d/
  2. Create the File laravel-worker.conf and put the following content
[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php $metager_path/artisan queue:work --sleep=1 --tries=1 --timeout=295 --daemon
autostart=true
autorestart=true
user=$username
numprocs=200
redirect_stderr=true
stdout_logfile=$metager_path/storage/logs/worker.log

=> Replace the $metager_path and set the user to the username you cloned MetaGer with 3. Restart/start the supervisor daemon sudo service supervisor start or sudo service supervisor restart => If it worked correctly you will see 200 php processes with pgrep php. You can change the number in the config to whatever you find reasonable. It defines how many requests can be made to your search engines in parallel.

Setup and configure Apache

  1. cd /etc/apache2/sites-available/
  2. Edit 000-default.conf and put the following content
<VirtualHost *:80>
	# If you have a DNS-Entry pointing to your machine you can comment out the following line and set the dns name there
        #ServerName www.example.com 

        ServerAdmin webmaster@localhost

	# Replace $metager_path with the actual path
        DocumentRoot $metager_path/public
	<Directory $metager_path/public>
		Allow Override all
		Require all granted
	</Directory>

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

</VirtualHost>
  1. sudo a2enmod rewrite
  2. restart/start the apache server sudo service apache2 restart

If you want to modify/add searchengines to your MetaGer instance you will have to modify the search engine definition in $metager_path/config/sumas.xml and you will have to add a parser class for that search engine to $metager_path/app/Models/parserSkripte/. With the information from the sumas.xml MetaGer will make a request to all search engines with type=web. The result will be given to above mentioned parser class where you have to write the code that transforms the result from the search engine API to a php object that MetaGer can use later.

Clone repository
  • Ideen und Vorschläge für MetaGer
  • Setup Metager
  • aufsetzen einer metager entwicklungsumgebung
  • betrieb einer metager instanz
  • einfuhrung in laravel
  • Home
  • howtoticketsystem
  • laravel elixir setup
  • suchmaschinenubersicht
  • sumas xml
  • usability
  • wie funktioniert metager