![]() |
#1 |
Junior Member
Join Date: Feb 2014
Posts: 26
|
Banning Streamrippers
I notice that my station gets "listeners" whose User Agent under Current Listener List is listed as "EMPTY". When I kick them, they come right back within seconds. I then will ban them, and see in the log that they try to reconnect for a long period of time. Is the "EMPTY" User Agent a definite streamripper, as when I ban them they keep trying to reconnect for hours?
Also, what is "StreamScraper"? Someone opened three separate streams with that listed, all from the same IP. Is that suspicious? |
![]() |
![]() |
#2 |
Join Date: Sep 2003
Posts: 27,873
|
not necessarily as it was found during internal testing that banning clients with no user-agent broke some legitimate clients from connecting (a hardware relayer if i remember correctly).
a streamripper is where a client connects and instead of just playing the stream, it also saves the audio data and depending on what is used, can effectively rip copies of the songs played as separate audio files. |
![]() |
![]() |
#3 | |
Junior Member
Join Date: Jan 2014
Posts: 47
|
Quote:
|
|
![]() |
![]() |
#4 |
Junior Member
Join Date: Apr 2006
Location: Somewhere under the sun
Posts: 34
|
As long as the client can see or hear it they can record it. There is little that can be done about this. The only way to fight streamrippers is to make it undesirable. Things like cross fading, embedding promos in the music, etc.
This post from 2004 is still relevant on the issue: http://forums.radiotoolbox.com/viewtopic.php?f=8&t=667 [edit - Moderation] other links provided in the post have been removed. |
![]() |
![]() |
#5 |
Junior Member
Join Date: Feb 2014
Posts: 26
|
Every "listener" that shows up with an empty user agent is coming from different German IP's. When I ban them, they try to reconnect for hours. Another thread I found mentioned this same problem, all with IP's coming from Germany. Is anyone else experiencing this? I ban one, then a different German IP shows up. After the ban, I can see the IP in the log attempting to reconnect for hours. If it were a real listener, they would give up fairly quickly. I don't understand why these IP's are all coming from Germany.
|
![]() |
![]() |
#6 |
Join Date: Sep 2003
Posts: 27,873
|
is probably looking like its mainly from Germany due to the large amount of hosting (non-SHOUTcast based) which is based there. and really its a failing battle to deal with things at the cost of real listeners (though I'm sure some stations like all of the dubious connections as it helps bump their station up the listings).
|
![]() |
![]() |
#7 |
Major Dude
Join Date: Mar 2011
Posts: 576
|
I have same here with german IPs and a empty user-agent. I don't believe that these are mobile phones if using the wrong link, it just starts downloading an infinite mp3. I have it tried with my galaxy-tab 3 and i've got a correct user-agent while downloading the infinite mp3.
|
![]() |
![]() |
#8 |
Join Date: Sep 2003
Posts: 27,873
|
maybe when DNAS development recommences, auto-blocking of connections from empty user-agents can be re-enabled (as there is handling already in place but was left disabled after it blocking legitimate connections).
also with that, hopefully other auto-banning improvements can be added e.g. country based filtering (which had been implemented internally but had not enabled due to licensing issues at the time) and i guess additional user-agent banning could be enabled as well. |
![]() |
![]() |
#9 |
Senior Member
Join Date: Nov 2010
Posts: 152
|
ban their ip in the webhosts control panel
|
![]() |
![]() |
#10 | |
Junior Member
Join Date: Feb 2014
Posts: 26
|
Quote:
It would be nice if Radionomy opens the source up to the DNAS and transcoder, so the community can take over or aid development. The open source software model just works better for software titles which attain interest, which in this case is true. Compare the Windows kernel to Linux: Dave Cutler wrote a decent kernel, but it doesn't evolve, change, improve, and have features added nearly as quickly as the largest software development project in the world, the Linux kernel. |
|
![]() |
![]() |
#11 | ||
Join Date: Sep 2003
Posts: 27,873
|
Quote:
Quote:
additionally, the biggest hurdle with DNAS development (other than dev(s) having the time to do it - which hopefully should be better with the new setup once things are put in place) is stations not wanting to / being bothered about updating. as apathy in development updates or just a complete refusal to update (as too much is focused on preserving up times and peak listener numbers for example) can put a complete dampener on dev(s) even wanting to work on the tools (i know i was at times under the previous ownership when time was allocated for updates when people weren't bothering / wanting to update). plus it's not like there aren't open source equivalents of the DNAS and transcoder already available for people wanting such things ![]() |
||
![]() |
![]() |
#12 |
Senior Member
|
Here is food for thought about Stream Rippers. I've experimented with one ripper (No I won't publish the name) where it asks "what agent do you want to mimic?" there I typed in iTunes/11.1.5 and tuned into my own station and I was able to see the app in the log. I then tried a few other apps like Winamp/5.581 and what do you know tuned to my station and there was Winamp/5.581 with my own IP address. A good streamripper can hide itself as any app even Mozilla/5.0. You could do the baby childish thing and have a random "your rocking with the legacy" go off in the background of the music say every 30-60 seconds (Like Q-106 in Lansing, Mi) does on new songs if your feeling high strung enough and your probably gonna get away with it if your Top40 or Rap. But Country, Progressive Rock, Metal you'll lose your listeners faster than you can say What??? (you know the new term these days.
I'll take advantage of when people record my station and after every third song have a file display in their player like this: ID - thelegacy.shorturl.com or sometimes ID - thelegacy dot shorturl dot com so it don't crash their ripper. This allows me to advertise my station because they'll have all sorts of files on their HDD with my station's website. you could follow it by 1-1000 so even if their ripper has the do not duplicate it will surely have 100's of files with your station website. They'll want to actually listen and because of all the files maybe not record so much. If they actually record they have to listen to pay attention. Use this to your advantage not as a thorn I say. Great Broadcasting Software Windows XP/7/8 http://nextkast.com For Progressive Rock, Classic Rock http://thelegacy.shorturl.com |
![]() |
![]() |
#13 |
Senior Member
Join Date: Aug 2001
Posts: 114
|
Yes, just like with web browsers & crawlers, there's no way to know if they are really using the software they claim to be.
And from what I've seen and read, all these German/Polish/Ukrainian/etc. streamripper operators absolutely don't care if there are no song titles, you're talking over everything, injecting commercials, whatever. They aren't even paying attention to what they are getting. I don't really understand it. |
![]() |
![]() |
#14 |
Senior Member
|
I know bandwidth can be a problem for some especially in the old days when I started and only had x number of slots. When I saw a listener that was on for 1 week non stop or more I would write down their IP and ban it for a bit. Then go back to my list and un ban it. It took some time before I found who the troublesome folks were. Some when I banned an ip would come right back as another IP and listen for non stop. This happened with a lot of overseas IP's. Lots of them were German and Japen and Russia. And for the record most of the Winamp 5.0 agents were a dead give away. You could ban all older Winamp apps that say 5.0. Anything greater than 5.0 let in. It could kill a few legit listeners (but very few). You'll see a lot less folks on for hours that way.
I see a lot more streamripper activity during the Summer and the holidays. So they will happen. On another note you could just sign up for Radionomy when Shoutcast is on their site and never worry about it since the burden would be upon them and it would actually count towards your revenue. Just food for thought. Great Broadcasting Software Windows XP/7/8 http://nextkast.com For Progressive Rock, Classic Rock http://thelegacy.shorturl.com |
![]() |
![]() |
#15 |
Junior Member
Join Date: Feb 2014
Posts: 26
|
I'm still having the German IP's show up; I ban one, and another connects. It's endless. It seems like somebody is using some type of proxy to connect. As of now, we have over 100 IP's in the ban list, and most all of them are German IP's. The functionality for the DNAS to automatically ban IP's based upon a predefined User Agent would definitely be helpful. Either someone really wants to record our stream, it's a game for somebody, or it's some type of bot. But the last option really would make no sense as the person wouldn't have much to gain from recording our stream for nothing.
|
![]() |
![]() |
#16 |
Forum King
Join Date: May 2009
Location: No longer on the streets of Kings County, CA.
Posts: 3,219
|
Hosting a DNAS on a linux server can be helpfull 'cause linux normally comes with the iptables firewall (or a compatible equivalent). When I was broadcasting, I did multiple things to stop stream rippers.
|
![]() |
![]() |
#17 | ||
Junior Member
Join Date: Feb 2014
Posts: 26
|
I'll try some of the things you recommended, although we use BSD. We are not on port 8000, but somewhere between 8000 and 9000. I'm not sure what you mean here:
Quote:
Quote:
|
||
![]() |
![]() |
#18 | |
Forum King
Join Date: May 2009
Location: No longer on the streets of Kings County, CA.
Posts: 3,219
|
Quote:
In the context of websites a playlist is just a downloadable resource, a file, not a webpage to be crawled by a search engine's bot/crawler. They typically have the extension m3u(8), pls, asx, ram, or whatnot and they have a doc/mime type that is not html, ergo, not downloaded, parsed, processed, and then searched for other online resources (i.e. "links") by search engine bots/crawlers. They do not download playlist files. If they don't download them, then they won't ever see the reference to your DNAS. By only sharing your DNAS's location with a playlist file then search engines won't see your DNAS, only the listening clients that are programmed to parse playlist files will see it. If search engines can't directly index your DNAS then they won't show up on search engines for the automated streamripper bots to find very easily. When I had a SHOUTcast station going not once did any search engine download my playlist files from my site. Yes, I checked my server logs. Daily actually. Every now-and-then I might get a "HEAD" request on one of them but that's usually done by the bot to check the mime/type of the resource, but it's not a download. I had multiple playlist files that I kept on my site for the different bitrates I was serving. Nowhere on my site pages were there any <a></a> tags with my DNAS port in them. For example, this did not exist on any of my pages: <a href="http://www.myserver.com:8000/listen96.aac">Listen</a>. I only had links like this: <a href="listen96.m3u">Listen</a>. Only inside the playlist files that I was hosting on my website did I have reference to the DNAS with the actual port number in it. Here's some example searches for DNAS servers that have been indexed: http://www.google.com/#q="SHOUTcast+...e%3Awinamp.com http://www.google.com/#q="SHOUTcast+...e%3Awinamp.com http://www.google.com/#q="SHOUTcast+...e%3Awinamp.com http://www.google.com/#q="Server is ...e%3Awinamp.com http://www.google.com/#q="Server is ...e%3Awinamp.com Botlicious yummy yum time! From a Google search of some fairly simple search terms, it's easy to find a number of DNAS servers indexed by the Google search engine. It's not as bad as it used to be before the recent robots.txt implementation in the SHOUTcast DNAS v2 but it's not pretty either. The mimetype of an HTML resource is text/html. Here's a website that shows the mimetypes of some of the common playlist formats: http://lizzy.sourceforge.net/docs/formats.html Here's another site, and it boasts a complete list: http://www.sitepoint.com/web-foundat...complete-list/ |
|
![]() |
![]() |
#19 |
Junior Member
Join Date: Feb 2014
Posts: 26
|
We're using the MixStream player which requires you to have a direct link to the DNAS. It won't work with a .pls file. Do you know of any alternatives, or if there's a way to hide or obfuscate the port number? We want to have a Flash or HTML5 player on our site so people can listen directly from there.
|
![]() |
![]() |
#20 |
Forum King
Join Date: May 2009
Location: No longer on the streets of Kings County, CA.
Posts: 3,219
|
|
![]() |
![]() |
#21 | |
Junior Member
Join Date: Feb 2014
Posts: 26
|
Quote:
|
|
![]() |
![]() |
#22 |
Forum King
Join Date: May 2009
Location: No longer on the streets of Kings County, CA.
Posts: 3,219
|
That's unfortunate, it doesn't do AAC.
I tested it with a local server and found that you can obfuscate the port number by some simple addition. So if your port is 8000, create a variable to hold a small fake port number, then add to it until you get the actual port number. Then insert the variable into the flashvars assignment. This way the port number is not directly visible to at least dumb bots. Example: HTML Code:
<script> // MixStream Flash Player, http://mixstreamflashplayer.net/ var oport=1234; oport=oport+6766; var flashvars = {};flashvars.serverHost = "127.0.0.1:"+oport+"/stream/3/";flashvars.getStats = "1";flashvars.autoStart = "1";flashvars.textColour = "FFFFFF";flashvars.buttonColour = "808080";var params = {};params.bgcolor= "80FFFF"; </script> |
![]() |
![]() |
#23 | |
Junior Member
Join Date: Feb 2014
Posts: 26
|
Thanks for your help. That worked perfectly. One last thing: Google indexed our old port number, so I created a robots.txt file to prevent the new one from being indexed. Can you please check if this looks right?
Quote:
|
|
![]() |
![]() |
#24 | |
Forum King
Join Date: May 2009
Location: No longer on the streets of Kings County, CA.
Posts: 3,219
|
The robots.txt specification does not work for sites on different ports. You would have to serve the robots.txt from the DNAS port. But, if you're using version SHOUTcast DNAS 2 RC (Build 27) 07/18/2011 or later of the DNAS then you don't have to worry about it as those already serve the correct robots.txt file. You can verify it by going to http://siteexample.com:69696/robots.txt
You should see something like the following: Quote:
|
|
![]() |
![]() |
#25 | |
Junior Member
Join Date: Feb 2014
Posts: 26
|
I added the robotstxtfile parameter to the DNAS config file, specifying the path to where the robots.txt file resides. One question, though, is that all the documentation I found pertaining to creating robots.txt files talks about excluding certain directories that obviously begin with a forward slash, and not beginning with a colon -- which is what I did as you can see above. Do you know if this will work or not?
An example of a standard robots.txt file is seen below: Quote:
|
|
![]() |
![]() |
#26 |
Forum King
Join Date: May 2009
Location: No longer on the streets of Kings County, CA.
Posts: 3,219
|
Actually, the default robots.txt that the DNAS generates if you don't provide one blocks everything. You shouldn't need to specify one for this situation.
As far as I know, in a robots.txt file all resources start with the forward slash to indicate server path root, the first directory. Here's some light reading on the subject: http://www.robotstxt.org/orig.html I'm at work right now so replies will come in slow if you have more questions. |
![]() |
![]() |
#27 |
Forum King
Join Date: Sep 2003
Location: Harpurhey, Manchester UK
Posts: 4,588
|
I've actually found a great way of thwarting steamrippers
I use Altacast (A Fork of Edcast) to stream with and set the metadata to update on 35 second interval. That way the song title doesn't change when the song does. Don't seem to have a problem with them. I know you will never stop them from trying but it's a start. |
![]() |
![]() |
#28 | |||
Junior Member
Join Date: Feb 2014
Posts: 26
|
Quote:
Quote:
Quote:
|
|||
![]() |
![]() |
#29 |
Join Date: Sep 2003
Posts: 27,873
|
just a / means ignore everything on the DNAS (since it's the base path of everything). though there's other ways which your stream could have been indexed (or it was done before using the newer DNAS since it's only present with some of the v2 DNAS builds and definitely not present with the v1.x DNAS).
|
![]() |
![]() |
#30 | |
Forum King
Join Date: Sep 2003
Location: Harpurhey, Manchester UK
Posts: 4,588
|
Quote:
None of the programs I use for streaming with have Spyware in them they are just full working free programs that do the job. I know of several shoutcast streaming programs that do report back to the company that owns them such as SAM with it's reg key system. Even simplecast did the same thing. I won't allow a program near my system if i think it's got spyware or other nasty malware in it. Learnt my lesson back in the early days of XP! |
|
![]() |
![]() |
#31 | |
Junior Member
Join Date: Feb 2014
Posts: 26
|
Thank you to thinktink and DrO for your help. Everything is up and running, and the stream ripping has stopped.
Quote:
![]() I'll take your word for it, but I don't think older versions of SAM phoned home. They did, however, hide multiple keys in the Windows registry which didn't get uninstalled upon uninstalling the program. That's how it knew when the trial expired. These days, it looks like most consumers are mainly interested in getting things through app stores, which means the Windows registry will most likely become deprecated at some point. Even though somebody like me is against a feudalistic and totalitarian app store that includes a remote kill switch in it (back door), I can see how consumers want the convenience. Gone will be the days of finagling with .msi files, cleaning up a bloated registry, and dealing with a system which slows down as time goes on. Maybe someday, Windows will just work. Me, on the other hand, will stick with Linux and BSD. Either the program controls the users, or the users control the program. I prefer the latter. |
|
![]() |
![]() |
#32 |
Forum King
Join Date: Sep 2003
Location: Harpurhey, Manchester UK
Posts: 4,588
|
How can i be sure I know the FREE programs I use don't have spyware in them? That's an easy question to answer.
I work closely with the developers of both Altacast and RadioDJ neither have spyware or any malicious code in them. That's how I know what's what! |
![]() |
![]() |
#33 | |
Junior Member
Join Date: Feb 2014
Posts: 26
|
Quote:
|
|
![]() |
![]() |
#34 |
Junior Member
Join Date: Jun 2014
Posts: 4
|
MusicDNA is the StreamRipper
hi, i too have the same issue with a blank field listener. after i tracert the ip (209.239.114.980) it goes to a company called 'musicdna-interface'
after looking up via search what 'musicdna' is i learn the are a for-profit commercial entity that, amongst other things, acts as an investigator and estimator of royalty payments. http://www.musicdna.com/en/our-service/index.html musicdna is ripping your broadcast and creating a library of music for a data base of monetization forecasting for perceived royalties. in my case, they are not listening to the broadcast as intended, but rather ripping my stream for their self promotion and profit. does anyone else have thoughts and/or opinions about this? |
![]() |
![]() |
#35 |
Junior Member
Join Date: Jun 2014
Posts: 4
|
MusicDNA breaking though my ban list
oddly, i banned the musicdna ip and subnet earlier this week and found they were back earlier today ripping my stream (for profit) and that they were somehow 'magically' off of my ban list.
i just re-banned their ip and subnet now, but has anyone else experienced a user breaking into their shoutcast admin portal and altering the ban list to re-enable their ip? |
![]() |
![]() |
#36 | |
Forum King
Join Date: Sep 2003
Location: Harpurhey, Manchester UK
Posts: 4,588
|
Quote:
Are you sure that IP is correct there is no .980 IP ranges only go from 0-255 |
|
![]() |
![]() |
#37 |
Junior Member
Join Date: Jun 2014
Posts: 4
|
good catch - ip is 209.239.114.98
|
![]() |
![]() |
#38 |
Junior Member
Join Date: Jun 2014
Posts: 4
|
so the musicdna-interface ip of 209.239.114.98 has breached my ipbanlist *AGAIN* and is back streamripping my music broadcast for their commercial gain.
any ideas how: a) they are 'unbanning' their ip from from my ban list? b) how to really ban them from my ban list? |
![]() |
![]() |
#39 |
Join Date: Sep 2003
Posts: 27,873
|
without knowing the DNAS version you're using, had to say.
|
![]() |
![]() |
#40 | |
Forum King
Join Date: Sep 2003
Location: Harpurhey, Manchester UK
Posts: 4,588
|
Quote:
|
|
![]() |
![]() |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|