ByteSpider

The ultimate meta forum: using a forum to talk about forums
Post Reply
User avatar
scooter
Administrator
Administrator
Ocean City
Posts: 3388
Joined: Tue Apr 21, 2020 8:38 pm
Location: Planet Earth, occasionally
House: Hufflepuff
Contact:

Fandoms

Arts & Crafts

Sports

Lifestyle

ByteSpider

Post by scooter »

I know you're reading this, so...is it really necessary to have five instances of your bot crawling all over my forum at the same time? :fu:

You need to check your algorithms.
This has been another bit of brain shrapnel from...
SCOOTER :cool:
Professional Dilettante (reasonable rates) :thumb:
skramer@atlantic-computingREMOVETHIS.com

QotM: "Never turn your back on fear. It should always be in front of you, like a thing that might have to be killed." – Hunter S. Thompson
User avatar
[Dimetrodon]
Moderator
Moderator
United States of America
Posts: 290
Joined: Sat Sep 19, 2020 8:45 pm
House: what

Fandoms

Lifestyle

Re: ByteSpider

Post by [Dimetrodon] »

Have you tried adding it as a bot?
User avatar
scooter
Administrator
Administrator
Ocean City
Posts: 3388
Joined: Tue Apr 21, 2020 8:38 pm
Location: Planet Earth, occasionally
House: Hufflepuff
Contact:

Fandoms

Arts & Crafts

Sports

Lifestyle

Re: ByteSpider

Post by scooter »

[Dimetrodon] wrote: Thu Jun 15, 2023 8:36 pm Have you tried adding it as a bot?
Done. It's Amazon's spider, and it is hammering the board of late. I had a new high of 21 "users" a few days ago and I'm betting that's what caused it.
This has been another bit of brain shrapnel from...
SCOOTER :cool:
Professional Dilettante (reasonable rates) :thumb:
skramer@atlantic-computingREMOVETHIS.com

QotM: "Never turn your back on fear. It should always be in front of you, like a thing that might have to be killed." – Hunter S. Thompson
User avatar
[Dimetrodon]
Moderator
Moderator
United States of America
Posts: 290
Joined: Sat Sep 19, 2020 8:45 pm
House: what

Fandoms

Lifestyle

Re: ByteSpider

Post by [Dimetrodon] »

scooter wrote: Sat Jun 17, 2023 2:36 pm
[Dimetrodon] wrote: Thu Jun 15, 2023 8:36 pm Have you tried adding it as a bot?
Done. It's Amazon's spider, and it is hammering the board of late. I had a new high of 21 "users" a few days ago and I'm betting that's what caused it.
Bizarrely enough, if the sid is removed from the url, it can cause multiple sessions to open up, which is what the bot was probably doing.

Most of those other sessions tend to be older and inactive with the bot inadvertently opening up a new session for every page it accesses. Now that it is a bot, that should be prevented.

If it accesses the board too many times to where it causes actual performance issues, you can block it: https://www.phpbb.com/community/viewtopic.php?t=2550501

Normally it is harmless, but some malicious clients have used the bot in a much more shady manner,, though my experiences with it have been benevolent.
User avatar
scooter
Administrator
Administrator
Ocean City
Posts: 3388
Joined: Tue Apr 21, 2020 8:38 pm
Location: Planet Earth, occasionally
House: Hufflepuff
Contact:

Fandoms

Arts & Crafts

Sports

Lifestyle

Re: ByteSpider

Post by scooter »

It's still hitting it regularly - seems to be the most active one out there - but I do notice fewer times when multiple sessions are going at the same time...
This has been another bit of brain shrapnel from...
SCOOTER :cool:
Professional Dilettante (reasonable rates) :thumb:
skramer@atlantic-computingREMOVETHIS.com

QotM: "Never turn your back on fear. It should always be in front of you, like a thing that might have to be killed." – Hunter S. Thompson
User avatar
scooter
Administrator
Administrator
Ocean City
Posts: 3388
Joined: Tue Apr 21, 2020 8:38 pm
Location: Planet Earth, occasionally
House: Hufflepuff
Contact:

Fandoms

Arts & Crafts

Sports

Lifestyle

Re: ByteSpider

Post by scooter »

scooter wrote: Sun Jun 18, 2023 1:35 pm It's still hitting it regularly - seems to be the most active one out there - but I do notice fewer times when multiple sessions are going at the same time...
"Regularly" seems to be something of an understatement. It's rare that I log in now and don't find ByteSpider logged in and scanning away. I can't figure how this is a sustainable algorithm.
This has been another bit of brain shrapnel from...
SCOOTER :cool:
Professional Dilettante (reasonable rates) :thumb:
skramer@atlantic-computingREMOVETHIS.com

QotM: "Never turn your back on fear. It should always be in front of you, like a thing that might have to be killed." – Hunter S. Thompson
Post Reply