simon987/Discord-Channel-scraper
1
0
Fork 0
mirror of https://github.com/simon987/Discord-Channel-scraper.git synced 2025-04-04 08:53:02 +00:00
Code Issues Projects Releases Wiki Activity

Added option to skip downloading messages. Reaction emoji are saved. Updated README

Browse Source
This commit is contained in:
Simon 2018-12-01 14:13:54 -05:00
parent 61b7a12d8f
commit 882112ca47
2 changed files with 15 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
README.md
View File

@ -19,6 +19,8 @@ $ python3.5 scrape-logs.py --help
usage: scrape-logs.py [-h] [--username USERNAME] [--flag FLAG] [--quiet]
[--server SERVER] [--channel CHANNEL] [--limit LIMIT]
[--output OUTPUT] [--logging {10,20,30,40,50}]
[--format FORMAT] [--dl_attachments] [--dl_emoji]
[--skip_messages]
Scrapes messages from a Discord channel.
@ -27,14 +29,14 @@ optional arguments:
--username USERNAME, -u USERNAME
Username to login under. If not specified, username
will be prompted for.
--flag FLAG, -f FLAG An alternative to specifing the server and channel,
--flag FLAG, -f FLAG An alternative to specifying the server and channel,
specify a piece of regex which when matched against a
message sent by the target user, will trigger scraping
of the channel the message was posted in. Useful for
private messages and private chats. Default value is
"!yank", activates by default if no server is
specified.
--quiet, -q Supress messages in Discord
--quiet, -q Suppress messages in Discord
--server SERVER, --guild SERVER, -s SERVER
Discord server name to scrape from (user must be a
member of the server and have history privileges).
@ -52,6 +54,11 @@ optional arguments:
<channel name>.txt.
--logging {10,20,30,40,50}
Change the logging level. Defaults to 20, info.
--format FORMAT, -F FORMAT
Message format (plain|json)
--dl_attachments, -a Download attachments
--dl_emoji, -e Download emoji
--skip_messages, -S Skip logging messages
```
## Future plans

7
scrape-logs.py
View File

@ -53,6 +53,7 @@ parser.add_argument('--logging', action='store', choices=[10, 20, 30, 40, 50], d
parser.add_argument('--format', '-F', action='store', default="plain", type=str, help='Message format (plain|json)')
parser.add_argument('--dl_attachments', '-a', action='store_true', help='Download attachments')
parser.add_argument('--dl_emoji', '-e', action='store_true', help='Download emoji')
parser.add_argument('--skip_messages', '-S', action='store_true', help='Skip logging messages')
args = parser.parse_args()
@ -137,13 +138,17 @@ async def get_logs(channel):
log.info("Getting the logs for channel {0}".format(channel.name))
with open("{0}.txt".format(channel.name), 'w') as f:
async for line in client.logs_from(channel, limit=args.limit):
save_line(f, line)
if not args.skip_messages:
save_line(f, line)
if args.dl_attachments:
for a in line.attachments:
download_attachment(a, line.channel.name)
if args.dl_emoji:
for e in EMOJI_RE.findall(line.content):
download_emoji(e)
for r in line.reactions:
if not isinstance(r.emoji, str):
download_emoji((r.emoji.name, r.emoji.id))
if not args.quiet:
await client.send_message(channel, 'The messages for this channel have been saved.')