Torn User Activity Scraper

This project is a web application that scrapes user activity data from the Torn API and displays the results. It includes features for starting and stopping the scraping process, viewing logs, and downloading results.

Features

Start and stop scraping user activity data
View real-time logs
Download data and log files
View scraping results and statistics
Toggle between light and dark mode

Requirements

Python 3.8+
Flask
Flask-Bootstrap
Flask-WTF
Pandas
Requests

Installation

Clone the repository:

git clone https://github.com/yourusername/torn-user-activity-scraper.git
cd torn-user-activity-scraper

Create a virtual environment and activate it:

python3 -m venv venv
source venv/bin/activate

Install the required packages:

pip install -r requirements.txt

Set up your configuration file: Create a config.ini file in the root directory of the project by renaming example_config.ini with the following content:

[DEFAULT]
SECRET_KEY = your_secret_key
API_KEY = your_api_key

[LOGGING]
VIEW_MAX_LINES = 500
VIEW_PAGE_LINES = 50
TAIL_PAGE_SIZE = 100 
LOG_DIR = log/

[BOOTSTRAP]
BOOTSTRAP_SERVE_LOCAL = False
BOOTSTRAP_BTN_STYLE = 'primary'
BOOTSTRAP_BTN_SIZE = 'sm'
BOOTSTRAP_ICON_SIZE = '1em'
BOOTSTRAP_ICON_COLOR = None
BOOTSTRAP_BOOTSWATCH_THEME = litera
BOOTSTRAP_MSG_CATEGORY = 'primary'
BOOTSTRAP_TABLE_VIEW_TITLE = 'View'
BOOTSTRAP_TABLE_EDIT_TITLE = 'Edit'
BOOTSTRAP_TABLE_DELETE_TITLE = 'Delete'
BOOTSTRAP_TABLE_NEW_TITLE = 'New'
BOOTSTRAP_FORM_GROUP_CLASSES = 'mb-3'
BOOTSTRAP_FORM_INLINE_CLASSES = 'row row-cols-lg-auto g-3 align-items-center'

[DATA]
DATA_DIR = data/

[TEMP]
TEMP_DIR = temp/

Usage

Run the Flask application:

flask run

Open your web browser and navigate to http://127.0.0.1:5000/.

License

This project is licensed under the MIT License.

1.9 KiB Raw Blame History