Quick links:  Login  |  Sign up  |  Site Rules  |  Support TMLfans

Media Thread

Started by Deebo, June 18, 2019, 10:55:08 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Guilt Trip

Quote from: Joe S. on March 16, 2022, 03:47:36 PM
I suppose. His show is on when I'm normally working, I just happen to be on vacation today and was in the car, but given the choice I'll actively avoid him. If he comes on another show as the 'hockey insider' I'll turn it off.
He's an idiot..pretend insider. I like Justin Bourne but still won't listen to Krapeos

Significantly Insignificant

#301
Not strictly media related, and I didn't really feel like starting an "I'm a dumbass" thread, but I think I got my IP banned from Natural Stat Trick.

I was trying to research my theory on how much Matthews defensive prowess contributes to wins by looking at his advanced stats and compare them between wins and loses.  The challenge there is that Natural Stat Trick doesn't have an easy way to produce data for individual games.  You would have to enter each date manually and get the data.

I figured I could do it quicker if I just wrote a python script to scrape the data from the site.  I pull the schedule from the NHL site, parse it, get the win and loses, and then grab the dates for each of those games.  I then try and pull the data from Natural Stat Trick using a per date query for the Leafs.  It worked once, and I was really excited.

However, since my first run, I can no longer access Natural Stat Trick from my IP address.  I can access it from my phone if I  am on cell, but not if I am on my wireless.  This is why I think my IP has been banned.  Either that or the Python module is holding something open, but that wouldn't be network wide.

Next step, IP cycling on my router :-).   
"We can't change what's done, we can only move on." - Arthur Morgan

CarltonTheBear

Quote from: Significantly Insignificant on April 23, 2022, 08:02:06 PM
Not strictly media related, and I didn't really feel like starting an "I'm a dumbass" thread, but I think I got my IP banned from Natural Stat Trick.

I was trying to research my theory on how much Matthews defensive prowess contributes to wins by looking at his advanced stats and compare them between wins and loses.  The challenge there is that Natural Stat Trick doesn't have an easy way to produce data for individual games.  You would have to enter each date manually and get the data.

I figured I could do it quicker if I just wrote a python script to scrape the data from the site.  I pull the schedule from the NHL site, parse it, get the win and loses, and then grab the dates for each of those games.  I then try and pull the data from Natural Stat Trick using a per date query for the Leafs.  It worked once, and I was really excited.

However, since my first run, I can no longer access Natural Stat Trick from my IP address.  I can access it from my phone if I  am on cell, but not if I am on my wireless.  This is why I think my IP has been banned.  Either that or the Python module is holding something open, but that wouldn't be network wide.

Next step, IP cycling on my router :-).   

Likely banned yeah. I vaguely recall them mentioning on twitter about how they used to let people scrap their data a certain amount of times during a certain time period but they had to drop that because it was just happening way too often and I guess that was affecting their performance. You could try reaching out to them on twitter and explaining the situation to see what they'd say.

An alternative, if/when you get back in, is exporting the data to excel or excel alternative and playing with it there.

Significantly Insignificant

Quote from: CarltonTheBear on April 23, 2022, 08:18:07 PM
Quote from: Significantly Insignificant on April 23, 2022, 08:02:06 PM
Not strictly media related, and I didn't really feel like starting an "I'm a dumbass" thread, but I think I got my IP banned from Natural Stat Trick.

I was trying to research my theory on how much Matthews defensive prowess contributes to wins by looking at his advanced stats and compare them between wins and loses.  The challenge there is that Natural Stat Trick doesn't have an easy way to produce data for individual games.  You would have to enter each date manually and get the data.

I figured I could do it quicker if I just wrote a python script to scrape the data from the site.  I pull the schedule from the NHL site, parse it, get the win and loses, and then grab the dates for each of those games.  I then try and pull the data from Natural Stat Trick using a per date query for the Leafs.  It worked once, and I was really excited.

However, since my first run, I can no longer access Natural Stat Trick from my IP address.  I can access it from my phone if I  am on cell, but not if I am on my wireless.  This is why I think my IP has been banned.  Either that or the Python module is holding something open, but that wouldn't be network wide.

Next step, IP cycling on my router :-).   

Likely banned yeah. I vaguely recall them mentioning on twitter about how they used to let people scrap their data a certain amount of times during a certain time period but they had to drop that because it was just happening way too often and I guess that was affecting their performance. You could try reaching out to them on twitter and explaining the situation to see what they'd say.

An alternative, if/when you get back in, is exporting the data to excel or excel alternative and playing with it there.

I can put a timeout into my script to see if that helps.  If it's time based, if I make the query happen every ten minutes or something I would hope that would get me around the ban.  Once I have the data, I'm good to go.  It'll just take a while to get the data.

I think I would have the same problem exporting to excel.  It's the presentation that natural stat trick provides.  It gives to a sum for a range of dates.  I want the data for individual games so that I can see if the data correlates in a particular way to a loss or a win. 

I'll reach out if I am interested in going forward with it some more.   I think if I set up a Patreon donation and let them know my intentions, maybe I can get an exemption or something.
"We can't change what's done, we can only move on." - Arthur Morgan

CarltonTheBear

Quote from: Significantly Insignificant on April 23, 2022, 08:25:05 PM
I can put a timeout into my script to see if that helps.  If it's time based, if I make the query happen every ten minutes or something I would hope that would get me around the ban.  Once I have the data, I'm good to go.  It'll just take a while to get the data.

I think I would have the same problem exporting to excel.  It's the presentation that natural stat trick provides.  It gives to a sum for a range of dates.  I want the data for individual games so that I can see if the data correlates in a particular way to a loss or a win. 

I'll reach out if I am interested in going forward with it some more.   I think if I set up a Patreon donation and let them know my intentions, maybe I can get an exemption or something.

Yeah checking what their exact policy on scraping would be a good first step. I doubt they'd have a problem with what you're doing exactly, IIRC they were seeing a massive amount of scrapes per day from what they believed were people using their data for gambling purposes.

Significantly Insignificant

Quote from: CarltonTheBear on April 23, 2022, 08:27:13 PM
Quote from: Significantly Insignificant on April 23, 2022, 08:25:05 PM
I can put a timeout into my script to see if that helps.  If it's time based, if I make the query happen every ten minutes or something I would hope that would get me around the ban.  Once I have the data, I'm good to go.  It'll just take a while to get the data.

I think I would have the same problem exporting to excel.  It's the presentation that natural stat trick provides.  It gives to a sum for a range of dates.  I want the data for individual games so that I can see if the data correlates in a particular way to a loss or a win. 

I'll reach out if I am interested in going forward with it some more.   I think if I set up a Patreon donation and let them know my intentions, maybe I can get an exemption or something.

Yeah checking what their exact policy on scraping would be a good first step. I doubt they'd have a problem with what you're doing exactly, IIRC they were seeing a massive amount of scrapes per day from what they believed were people using their data for gambling purposes.

I am going to send them a message, but I have a workaround that I can use until they get back to me.
"We can't change what's done, we can only move on." - Arthur Morgan

Zee

https://twitter.com/OffsideDH/status/1534197019287224322?s=20&t=BUwbZOtEyDWq4KbhU7HJpg

Eh, this will get more play solely based on whose son he is.  Not a good look.

LittleHockeyFan

uh, whose son is he? I'm unfamiliar.

Love all the typos, bad grammar and spelling mistakes in the arrest report. :D

Deebo

Quote from: LittleHockeyFan on June 07, 2022, 03:04:15 PM
uh, whose son is he? I'm unfamiliar.

Love all the typos, bad grammar and spelling mistakes in the arrest report. :D

Bob McKenzie

Heroic Shrimp

Quote from: Deebo on June 07, 2022, 03:38:43 PM
Quote from: LittleHockeyFan on June 07, 2022, 03:04:15 PM
uh, whose son is he? I'm unfamiliar.

Love all the typos, bad grammar and spelling mistakes in the arrest report. :D

Bob McKenzie
Supersize my Freedom Fries

Frank E

Take a bow, HS.

Awesome.

Bender

Quote from: Zee on June 07, 2022, 01:54:16 PM
https://twitter.com/OffsideDH/status/1534197019287224322?s=20&t=BUwbZOtEyDWq4KbhU7HJpg

Eh, this will get more play solely based on whose son he is.  Not a good look.
I think this looks worse on law enforcement. My brain can't handle that level of incorrect grammar.
"They say you can judge a man by the company he keeps. So here is the professor's oldest friend, a grotesque, stinking lobster." - Bender

WhatIfGodWasALeaf

I think he deserves a break, never got physical, probably a good idea for him to make some lifestyle changes and apologize publicly.


Heroic Shrimp

Supersize my Freedom Fries

Zee

Quote from: Heroic Shrimp on June 07, 2022, 07:15:51 PM
https://twitter.com/dalter/status/1534300075907768320

Lol, no respect for Hack Todd.

This made me smile as well.  Shades of Damian Cox surfing for porn on a Sunday morning and forgetting his open browser tabs while posting a screenshot on twitter.