93 lines
1.9 KiB
Markdown
93 lines
1.9 KiB
Markdown
# Torn User Activity Scraper
|
|
|
|
This project is a web application that scrapes user activity data from the Torn API and displays the results. It includes features for starting and stopping the scraping process, viewing logs, and downloading results.
|
|
|
|
## Features
|
|
|
|
- Start and stop scraping user activity data
|
|
- View real-time logs
|
|
- Download data and log files
|
|
- View scraping results and statistics
|
|
- Toggle between light and dark mode
|
|
|
|
## Requirements
|
|
|
|
- Python 3.8+
|
|
- Flask
|
|
- Flask-Bootstrap
|
|
- Flask-WTF
|
|
- Pandas
|
|
- Requests
|
|
|
|
## Installation
|
|
|
|
1. Clone the repository:
|
|
|
|
```sh
|
|
git clone https://github.com/yourusername/torn-user-activity-scraper.git
|
|
cd torn-user-activity-scraper
|
|
```
|
|
|
|
2. Create a virtual environment and activate it:
|
|
|
|
```sh
|
|
python3 -m venv venv
|
|
source venv/bin/activate
|
|
```
|
|
|
|
3. Install the required packages:
|
|
|
|
```sh
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
4. Set up your configuration file:
|
|
Create a `config.ini` file in the root directory of the project by renaming `example_config.ini` with the following content:
|
|
|
|
```ini
|
|
[DEFAULT]
|
|
SECRET_KEY = your_secret_key
|
|
API_KEY = your_api_key
|
|
|
|
[LOGGING]
|
|
VIEW_MAX_LINES = 500
|
|
VIEW_PAGE_LINES = 50
|
|
TAIL_PAGE_SIZE = 100
|
|
LOG_DIR = log/
|
|
|
|
[BOOTSTRAP]
|
|
BOOTSTRAP_SERVE_LOCAL = False
|
|
BOOTSTRAP_BTN_STYLE = 'primary'
|
|
BOOTSTRAP_BTN_SIZE = 'sm'
|
|
BOOTSTRAP_ICON_SIZE = '1em'
|
|
BOOTSTRAP_ICON_COLOR = None
|
|
BOOTSTRAP_BOOTSWATCH_THEME = litera
|
|
BOOTSTRAP_MSG_CATEGORY = 'primary'
|
|
BOOTSTRAP_TABLE_VIEW_TITLE = 'View'
|
|
BOOTSTRAP_TABLE_EDIT_TITLE = 'Edit'
|
|
BOOTSTRAP_TABLE_DELETE_TITLE = 'Delete'
|
|
BOOTSTRAP_TABLE_NEW_TITLE = 'New'
|
|
BOOTSTRAP_FORM_GROUP_CLASSES = 'mb-3'
|
|
BOOTSTRAP_FORM_INLINE_CLASSES = 'row row-cols-lg-auto g-3 align-items-center'
|
|
|
|
[DATA]
|
|
DATA_DIR = data/
|
|
|
|
[TEMP]
|
|
TEMP_DIR = temp/
|
|
```
|
|
|
|
## Usage
|
|
|
|
1. Run the Flask application:
|
|
|
|
```sh
|
|
flask run
|
|
```
|
|
|
|
2. Open your web browser and navigate to `http://127.0.0.1:5000/`.
|
|
|
|
## License
|
|
|
|
This project is licensed under the MIT License.
|