Reading the repo
Reading the repo
How to install on Linux
Pipeline to find company websites, detect contact forms and load results into MySQL
Saved setup from 5 h ago
Will it work on Linux?
pip install selenium webdriver-manager requests beautifulsoup4 pydoll-python mysql-connector-python
Google Chrome is required for the Selenium and pydoll stages.
db_loader.py reads its connection settings from environment variables:
export DB_HOST=... DB_USER=... DB_PASSWORD=... DB_NAME=... DB_PORT=3306
Batch sizes, crawl depth and delays are set in the CONFIG block at the top of each script.
Then run bash setup.sh. It stops at the first error and is safe to run again. Servers and tests aren't started; it lists them at the end. Read it first.
git clone https://github.com/Hassan-as-DA/contact-form-submission-pipeline.git cd contact-form-submission-pipeline
Python 3
sudo apt install python3 python3-pip
In the maintainers' order. Do the README instructions above first — they cover steps outside the terminal.
pip install selenium webdriver-manager requests beautifulsoup4 pydoll-python mysql-connector-python export DB_HOST=... DB_USER=... DB_PASSWORD=... DB_NAME=... DB_PORT=3306
No reports for Linux yet.
Opens a chat with this manual already filled in, for Linux. It uses your own account; gitmanual sends nothing.