Old 20th February 2014, 14:53   #1
pirithous
Junior Member
 
Join Date: Feb 2014
Posts: 26
Banning Streamrippers

I notice that my station gets "listeners" whose User Agent under Current Listener List is listed as "EMPTY". When I kick them, they come right back within seconds. I then will ban them, and see in the log that they try to reconnect for a long period of time. Is the "EMPTY" User Agent a definite streamripper, as when I ban them they keep trying to reconnect for hours?

Also, what is "StreamScraper"? Someone opened three separate streams with that listed, all from the same IP. Is that suspicious?
pirithous is offline   Reply With Quote
Old 20th February 2014, 15:02   #2
DrO
 
Join Date: Sep 2003
Posts: 27,873
not necessarily as it was found during internal testing that banning clients with no user-agent broke some legitimate clients from connecting (a hardware relayer if i remember correctly).

a streamripper is where a client connects and instead of just playing the stream, it also saves the audio data and depending on what is used, can effectively rip copies of the songs played as separate audio files.
DrO is offline   Reply With Quote
Old 20th February 2014, 17:21   #3
aron9forever
Junior Member
 
Join Date: Jan 2014
Posts: 47
Quote:
Originally Posted by DrO View Post
not necessarily as it was found during internal testing that banning clients with no user-agent broke some legitimate clients from connecting (a hardware relayer if i remember correctly).

a streamripper is where a client connects and instead of just playing the stream, it also saves the audio data and depending on what is used, can effectively rip copies of the songs played as separate audio files.
I've seen this thing happening for example on some mobile phones if using the wrong link, it just starts downloading an infinite mp3
aron9forever is offline   Reply With Quote
Old 21st February 2014, 10:11   #4
Kigen
Junior Member
 
Join Date: Apr 2006
Location: Somewhere under the sun
Posts: 34
As long as the client can see or hear it they can record it. There is little that can be done about this. The only way to fight streamrippers is to make it undesirable. Things like cross fading, embedding promos in the music, etc.

This post from 2004 is still relevant on the issue:
http://forums.radiotoolbox.com/viewtopic.php?f=8&t=667

[edit - Moderation]
other links provided in the post have been removed.
Kigen is offline   Reply With Quote
Old 26th February 2014, 22:03   #5
pirithous
Junior Member
 
Join Date: Feb 2014
Posts: 26
Every "listener" that shows up with an empty user agent is coming from different German IP's. When I ban them, they try to reconnect for hours. Another thread I found mentioned this same problem, all with IP's coming from Germany. Is anyone else experiencing this? I ban one, then a different German IP shows up. After the ban, I can see the IP in the log attempting to reconnect for hours. If it were a real listener, they would give up fairly quickly. I don't understand why these IP's are all coming from Germany.
pirithous is offline   Reply With Quote
Old 27th February 2014, 08:33   #6
DrO
 
Join Date: Sep 2003
Posts: 27,873
is probably looking like its mainly from Germany due to the large amount of hosting (non-SHOUTcast based) which is based there. and really its a failing battle to deal with things at the cost of real listeners (though I'm sure some stations like all of the dubious connections as it helps bump their station up the listings).
DrO is offline   Reply With Quote
Old 27th February 2014, 16:06   #7
neralex
Major Dude
 
Join Date: Mar 2011
Posts: 576
I have same here with german IPs and a empty user-agent. I don't believe that these are mobile phones if using the wrong link, it just starts downloading an infinite mp3. I have it tried with my galaxy-tab 3 and i've got a correct user-agent while downloading the infinite mp3.
neralex is offline   Reply With Quote
Old 27th February 2014, 16:12   #8
DrO
 
Join Date: Sep 2003
Posts: 27,873
maybe when DNAS development recommences, auto-blocking of connections from empty user-agents can be re-enabled (as there is handling already in place but was left disabled after it blocking legitimate connections).

also with that, hopefully other auto-banning improvements can be added e.g. country based filtering (which had been implemented internally but had not enabled due to licensing issues at the time) and i guess additional user-agent banning could be enabled as well.
DrO is offline   Reply With Quote
Old 27th February 2014, 17:24   #9
p0rt
Senior Member
 
Join Date: Nov 2010
Posts: 152
ban their ip in the webhosts control panel
p0rt is offline   Reply With Quote
Old 28th February 2014, 12:20   #10
pirithous
Junior Member
 
Join Date: Feb 2014
Posts: 26
Quote:
Originally Posted by DrO View Post
maybe when DNAS development recommences, auto-blocking of connections from empty user-agents can be re-enabled (as there is handling already in place but was left disabled after it blocking legitimate connections).

also with that, hopefully other auto-banning improvements can be added e.g. country based filtering (which had been implemented internally but had not enabled due to licensing issues at the time) and i guess additional user-agent banning could be enabled as well.
I'm banning "listeners" from Germany on a daily basis with an empty user agent, so this would be a nice feature. I can't figure out if it's someone who thinks it's funny to hog bandwidth, or if someone is trying to record the stream. Either way, most of this all comes from Germany, and whatever program they're using attempts to reconnect over and over again at exact intervals once the ban has taken place.

It would be nice if Radionomy opens the source up to the DNAS and transcoder, so the community can take over or aid development. The open source software model just works better for software titles which attain interest, which in this case is true. Compare the Windows kernel to Linux: Dave Cutler wrote a decent kernel, but it doesn't evolve, change, improve, and have features added nearly as quickly as the largest software development project in the world, the Linux kernel.
pirithous is offline   Reply With Quote
Old 28th February 2014, 12:51   #11
DrO
 
Join Date: Sep 2003
Posts: 27,873
Quote:
Originally Posted by pirithous View Post
I'm banning "listeners" from Germany on a daily basis with an empty user agent, so this would be a nice feature. I can't figure out if it's someone who thinks it's funny to hog bandwidth, or if someone is trying to record the stream. Either way, most of this all comes from Germany, and whatever program they're using attempts to reconnect over and over again at exact intervals once the ban has taken place.
unless you can physically track down who / what is causing it, who can really say what is the reason for such a specific (as such connections from Germany seem to be more of an annoyance than similar ones from China as was often a reported issue a few years ago - unless they're using German VPNs?).

Quote:
Originally Posted by pirithous View Post
It would be nice if Radionomy opens the source up to the DNAS and transcoder
open source is not a be all and wonderful thing that people want / expect it to be in most cases - yes it can have it's uses and can do good, but too often all i've seen is people go over the bones of something that was open sourced, get what they want and the original project then just dies. obviously that can be proven is not the case for all, but you'll have to forgive my scepticism on hearing that call to open source 'popular' projects and the original project never benefit and die.


additionally, the biggest hurdle with DNAS development (other than dev(s) having the time to do it - which hopefully should be better with the new setup once things are put in place) is stations not wanting to / being bothered about updating.

as apathy in development updates or just a complete refusal to update (as too much is focused on preserving up times and peak listener numbers for example) can put a complete dampener on dev(s) even wanting to work on the tools (i know i was at times under the previous ownership when time was allocated for updates when people weren't bothering / wanting to update).


plus it's not like there aren't open source equivalents of the DNAS and transcoder already available for people wanting such things
DrO is offline   Reply With Quote
Old 8th March 2014, 17:11   #12
voodoohippie
Senior Member
 
Join Date: Jun 2009
Location: Elizabeth City, NC
Posts: 214
Send a message via Yahoo to voodoohippie
Here is food for thought about Stream Rippers. I've experimented with one ripper (No I won't publish the name) where it asks "what agent do you want to mimic?" there I typed in iTunes/11.1.5 and tuned into my own station and I was able to see the app in the log. I then tried a few other apps like Winamp/5.581 and what do you know tuned to my station and there was Winamp/5.581 with my own IP address. A good streamripper can hide itself as any app even Mozilla/5.0. You could do the baby childish thing and have a random "your rocking with the legacy" go off in the background of the music say every 30-60 seconds (Like Q-106 in Lansing, Mi) does on new songs if your feeling high strung enough and your probably gonna get away with it if your Top40 or Rap. But Country, Progressive Rock, Metal you'll lose your listeners faster than you can say What??? (you know the new term these days.

I'll take advantage of when people record my station and after every third song have a file display in their player like this: ID - thelegacy.shorturl.com or sometimes ID - thelegacy dot shorturl dot com so it don't crash their ripper. This allows me to advertise my station because they'll have all sorts of files on their HDD with my station's website. you could follow it by 1-1000 so even if their ripper has the do not duplicate it will surely have 100's of files with your station website. They'll want to actually listen and because of all the files maybe not record so much. If they actually record they have to listen to pay attention. Use this to your advantage not as a thorn I say.

Great Broadcasting Software Windows XP/7/8
http://nextkast.com

For Progressive Rock, Classic Rock http://thelegacy.shorturl.com
voodoohippie is offline   Reply With Quote
Old 26th March 2014, 00:38   #13
mjbrown
Senior Member
 
Join Date: Aug 2001
Posts: 114
Yes, just like with web browsers & crawlers, there's no way to know if they are really using the software they claim to be.

And from what I've seen and read, all these German/Polish/Ukrainian/etc. streamripper operators absolutely don't care if there are no song titles, you're talking over everything, injecting commercials, whatever. They aren't even paying attention to what they are getting. I don't really understand it.
mjbrown is offline   Reply With Quote
Old 29th March 2014, 15:42   #14
voodoohippie
Senior Member
 
Join Date: Jun 2009
Location: Elizabeth City, NC
Posts: 214
Send a message via Yahoo to voodoohippie
I know bandwidth can be a problem for some especially in the old days when I started and only had x number of slots. When I saw a listener that was on for 1 week non stop or more I would write down their IP and ban it for a bit. Then go back to my list and un ban it. It took some time before I found who the troublesome folks were. Some when I banned an ip would come right back as another IP and listen for non stop. This happened with a lot of overseas IP's. Lots of them were German and Japen and Russia. And for the record most of the Winamp 5.0 agents were a dead give away. You could ban all older Winamp apps that say 5.0. Anything greater than 5.0 let in. It could kill a few legit listeners (but very few). You'll see a lot less folks on for hours that way.

I see a lot more streamripper activity during the Summer and the holidays. So they will happen.

On another note you could just sign up for Radionomy when Shoutcast is on their site and never worry about it since the burden would be upon them and it would actually count towards your revenue. Just food for thought.

Great Broadcasting Software Windows XP/7/8
http://nextkast.com

For Progressive Rock, Classic Rock http://thelegacy.shorturl.com
voodoohippie is offline   Reply With Quote
Old 29th April 2014, 17:07   #15
pirithous
Junior Member
 
Join Date: Feb 2014
Posts: 26
I'm still having the German IP's show up; I ban one, and another connects. It's endless. It seems like somebody is using some type of proxy to connect. As of now, we have over 100 IP's in the ban list, and most all of them are German IP's. The functionality for the DNAS to automatically ban IP's based upon a predefined User Agent would definitely be helpful. Either someone really wants to record our stream, it's a game for somebody, or it's some type of bot. But the last option really would make no sense as the person wouldn't have much to gain from recording our stream for nothing.
pirithous is offline   Reply With Quote
Old 30th April 2014, 15:24   #16
thinktink
Forum King
 
thinktink's Avatar
 
Join Date: May 2009
Location: On the streets of Kings County, CA.
Posts: 3,007
Send a message via Skype™ to thinktink
Hosting a DNAS on a linux server can be helpfull 'cause linux normally comes with the iptables firewall (or a compatible equivalent). When I was broadcasting, I did multiple things to stop stream rippers.
  1. Crossfade the crazy out of the music (this was actually the most effective), a minimum 10 second crossfade between songs.
  2. Don't directly link to the DNAS from your website, only provide a playlist with the stream information in it, not even the playlist from the DNAS. This way, the DNAS doesn't get crawled by search engine bots that don't know how to parse playlist files and then your DNAS pages don't show up on search engines for automated stream ripper bots to find and index.
  3. Block entire bad countries like China, Germany, Russia, and others right from iptables. You can find scripts online to do this for you automatically. IP address allocations change frequently so you have to re-run the script on a periodic bases.
  4. Do not use port 8000. Port 8000 is used for almost everything from alternative webservers, automation servers, proxy clients and servers, the list goes on. It's a very commonly used port. Select a very different port somewhere between 10000 and 65534.
  5. Setup rules in iptables to block multiple connections from the same IP. Sometimes this will block clients from being able to get metadata though this is a small number of clients.
thinktink is offline   Reply With Quote
Old 30th April 2014, 15:47   #17
pirithous
Junior Member
 
Join Date: Feb 2014
Posts: 26
I'll try some of the things you recommended, although we use BSD. We are not on port 8000, but somewhere between 8000 and 9000. I'm not sure what you mean here:

Quote:
Originally Posted by thinktink View Post
Don't directly link to the DNAS from your website, only provide a playlist with the stream information in it, not even the playlist from the DNAS. This way, the DNAS doesn't get crawled by search engine bots that don't know how to parse playlist files and then your DNAS pages don't show up on search engines for automated stream ripper bots to find and index.
Isn't a playlist with "the stream information in it" directly linking to the DNAS? Maybe you can be more clear. Below is a snippet of the DNAS log file, showing just a few minutes of activity. I'm not sure about banning entire countries. That sounds a bit over-the-top, because valid listeners will get banned too. For whatever reason, Germany is a major problem. I ban one IP, and within less than a minute another one shows up. And another, and another. It's either a bot or someone who's really bored. I can't figure out which.

Quote:
2014-04-30 11:21:24 INFO [YP] Updating listing details for stream #1 succeeded.
2014-04-30 11:21:26 INFO [BAN] Added `91.7.228.213/255' to global ban list
2014-04-30 11:21:27 INFO [188.101.192.62 sid=1] IP in banned list - disconnecting.
2014-04-30 11:21:43 INFO [188.193.34.135 sid=1] IP in banned list - disconnecting.
2014-04-30 11:21:58 INFO [188.101.192.62 sid=1] IP in banned list - disconnecting.
2014-04-30 11:22:07 INFO [BAN] Added `37.138.99.81/255' to global ban list
2014-04-30 11:22:08 INFO [DST 88.130.166.250 sid=1] SHOUTcast 1 client connection accepted. User-Agent: `mhttplib/streamWriter/5.0.0.0', UID: 684670592
2014-04-30 11:22:15 INFO [188.193.34.135 sid=1] IP in banned list - disconnecting.
2014-04-30 11:22:17 INFO [DST 88.130.166.250 sid=1] SHOUTcast 1 client connection closed (9 seconds) [Bytes: 167739] Agent: `mhttplib/streamWriter/5.0.0.0', UID: 684670592
2014-04-30 11:22:24 INFO [DST 92.224.133.57 sid=1] SHOUTcast 1 client connection accepted. User-Agent: `mhttplib/streamWriter/5.0.0.0', UID: 684670592
2014-04-30 11:22:46 INFO [188.193.34.135 sid=1] IP in banned list - disconnecting.
2014-04-30 11:22:54 INFO [BAN] Added `92.224.133.57/255' to global ban list
2014-04-30 11:22:54 INFO [DST 92.224.133.57 sid=1] Kicked
2014-04-30 11:22:54 INFO [DST 92.224.133.57 sid=1] SHOUTcast 1 client connection closed (30 seconds) [Bytes: 519328] Agent: `mhttplib/streamWriter/5.0.0.0', UID: 684670592
2014-04-30 11:23:10 INFO [BAN] `92.224.133.57' already in the global ban list
2014-04-30 11:23:18 INFO [188.193.34.135 sid=1] IP in banned list - disconnecting.
2014-04-30 11:23:25 INFO [92.224.133.57 sid=1] IP in banned list - disconnecting.
2014-04-30 11:23:38 INFO [BAN] Added `88.130.166.250/255' to global ban list
2014-04-30 11:23:49 INFO [188.193.34.135 sid=1] IP in banned list - disconnecting.
2014-04-30 11:23:56 INFO [92.224.133.57 sid=1] IP in banned list - disconnecting.
2014-04-30 11:24:10 INFO [91.7.228.213 sid=1] IP in banned list - disconnecting.
2014-04-30 11:24:21 INFO [188.193.34.135 sid=1] IP in banned list - disconnecting.
2014-04-30 11:24:31 INFO [91.7.228.213 sid=1] IP in banned list - disconnecting.
2014-04-30 11:24:52 INFO [DST 176.199.159.116 sid=1] SHOUTcast 1 client connection accepted. User-Agent: `mhttplib/streamWriter/5.0.0.0', UID: 684655232
2014-04-30 11:24:52 INFO [188.193.34.135 sid=1] IP in banned list - disconnecting.
2014-04-30 11:24:55 INFO [DST 176.199.159.116 sid=1] SHOUTcast 1 client connection closed (3 seconds) [Bytes: 75743] Agent: `mhttplib/streamWriter/5.0.0.0', UID: 684655232
2014-04-30 11:25:03 INFO [178.9.134.212 sid=1] IP in banned list - disconnecting.
2014-04-30 11:25:17 INFO [84.134.248.34 sid=1] IP in banned list - disconnecting.
2014-04-30 11:25:24 INFO [188.193.34.135 sid=1] IP in banned list - disconnecting.
2014-04-30 11:25:25 INFO [84.134.248.34 sid=1] IP in banned list - disconnecting.
2014-04-30 11:25:39 INFO [91.7.228.213 sid=1] IP in banned list - disconnecting.
2014-04-30 11:25:52 INFO [DST 79.232.204.55 sid=1] SHOUTcast 1 client connection accepted. User-Agent: `mhttplib/streamWriter/5.0.0.0', UID: 684655232
2014-04-30 11:25:55 INFO [DST 79.232.204.55 sid=1] SHOUTcast 1 client connection closed (3 seconds) [Bytes: 64763] Agent: `mhttplib/streamWriter/5.0.0.0', UID: 684655232
2014-04-30 11:25:55 INFO [BAN] Added `176.199.159.116/255' to global ban list
2014-04-30 11:25:55 INFO [188.193.34.135 sid=1] IP in banned list - disconnecting.
2014-04-30 11:26:10 INFO [DST 87.169.151.183 sid=1] SHOUTcast 1 client connection accepted. User-Agent: `mhttplib/streamWriter/5.0.0.0', UID: 684655232
2014-04-30 11:26:16 INFO [DST 87.169.151.183 sid=1] SHOUTcast 1 client connection closed (6 seconds) [Bytes: 53654] Agent: `mhttplib/streamWriter/5.0.0.0', UID: 684655232
2014-04-30 11:26:27 INFO [188.193.34.135 sid=1] IP in banned list - disconnecting.
2014-04-30 11:26:56 INFO [BAN] Added `79.232.204.55/255' to global ban list
2014-04-30 11:26:58 INFO [188.193.34.135 sid=1] IP in banned list - disconnecting.
pirithous is offline   Reply With Quote
Old 30th April 2014, 17:55   #18
thinktink
Forum King
 
thinktink's Avatar
 
Join Date: May 2009
Location: On the streets of Kings County, CA.
Posts: 3,007
Send a message via Skype™ to thinktink
Quote:
Originally Posted by pirithous View Post
...

Isn't a playlist with "the stream information in it" directly linking to the DNAS? ...
No, it isn't.


Quote:
Originally Posted by pirithous View Post
... Maybe you can be more clear.
In the context of websites a playlist is just a downloadable resource, a file, not a webpage to be crawled by a search engine's bot/crawler. They typically have the extension m3u(8), pls, asx, ram, or whatnot and they have a doc/mime type that is not html, ergo, not downloaded, parsed, processed, and then searched for other online resources (i.e. "links") by search engine bots/crawlers. They do not download playlist files. If they don't download them, then they won't ever see the reference to your DNAS. By only sharing your DNAS's location with a playlist file then search engines won't see your DNAS, only the listening clients that are programmed to parse playlist files will see it. If search engines can't directly index your DNAS then they won't show up on search engines for the automated streamripper bots to find very easily.

When I had a SHOUTcast station going not once did any search engine download my playlist files from my site. Yes, I checked my server logs. Daily actually. Every now-and-then I might get a "HEAD" request on one of them but that's usually done by the bot to check the mime/type of the resource, but it's not a download. I had multiple playlist files that I kept on my site for the different bitrates I was serving. Nowhere on my site pages were there any <a></a> tags with my DNAS port in them. For example, this did not exist on any of my pages: <a href="http://www.myserver.com:8000/listen96.aac">Listen</a>. I only had links like this: <a href="listen96.m3u">Listen</a>. Only inside the playlist files that I was hosting on my website did I have reference to the DNAS with the actual port number in it.

Here's some example searches for DNAS servers that have been indexed:
http://www.google.com/#q="SHOUTcast+...e%3Awinamp.com
http://www.google.com/#q="SHOUTcast+...e%3Awinamp.com
http://www.google.com/#q="SHOUTcast+...e%3Awinamp.com
http://www.google.com/#q="Server is ...e%3Awinamp.com
http://www.google.com/#q="Server is ...e%3Awinamp.com
Botlicious yummy yum time!
From a Google search of some fairly simple search terms, it's easy to find a number of DNAS servers indexed by the Google search engine. It's not as bad as it used to be before the recent robots.txt implementation in the SHOUTcast DNAS v2 but it's not pretty either.



The mimetype of an HTML resource is text/html.

Here's a website that shows the mimetypes of some of the common playlist formats:
http://lizzy.sourceforge.net/docs/formats.html

Here's another site, and it boasts a complete list:
http://www.sitepoint.com/web-foundat...complete-list/
thinktink is offline   Reply With Quote
Old 1st May 2014, 05:03   #19
pirithous
Junior Member
 
Join Date: Feb 2014
Posts: 26
We're using the MixStream player which requires you to have a direct link to the DNAS. It won't work with a .pls file. Do you know of any alternatives, or if there's a way to hide or obfuscate the port number? We want to have a Flash or HTML5 player on our site so people can listen directly from there.
pirithous is offline   Reply With Quote
Old 1st May 2014, 20:01   #20
thinktink
Forum King
 
thinktink's Avatar
 
Join Date: May 2009
Location: On the streets of Kings County, CA.
Posts: 3,007
Send a message via Skype™ to thinktink
Quote:
Originally Posted by pirithous View Post
We're using the MixStream player...
This one?

http://mixstream.net/streaming/creat...dio-player.php
thinktink is offline   Reply With Quote
Old 2nd May 2014, 03:13   #21
pirithous
Junior Member
 
Join Date: Feb 2014
Posts: 26
Quote:
Originally Posted by thinktink View Post
Yes, exactly.
pirithous is offline   Reply With Quote
Old 2nd May 2014, 06:22   #22
thinktink
Forum King
 
thinktink's Avatar
 
Join Date: May 2009
Location: On the streets of Kings County, CA.
Posts: 3,007
Send a message via Skype™ to thinktink
That's unfortunate, it doesn't do AAC.

I tested it with a local server and found that you can obfuscate the port number by some simple addition. So if your port is 8000, create a variable to hold a small fake port number, then add to it until you get the actual port number. Then insert the variable into the flashvars assignment. This way the port number is not directly visible to at least dumb bots.

Example:
HTML Code:
<script>
	// MixStream Flash Player, http://mixstreamflashplayer.net/
	var oport=1234;
	oport=oport+6766;
	var flashvars = {};flashvars.serverHost = "127.0.0.1:"+oport+"/stream/3/";flashvars.getStats = "1";flashvars.autoStart = "1";flashvars.textColour = "FFFFFF";flashvars.buttonColour = "808080";var params = {};params.bgcolor= "80FFFF";
</script>
thinktink is offline   Reply With Quote
Old 3rd May 2014, 07:12   #23
pirithous
Junior Member
 
Join Date: Feb 2014
Posts: 26
Thanks for your help. That worked perfectly. One last thing: Google indexed our old port number, so I created a robots.txt file to prevent the new one from being indexed. Can you please check if this looks right?

Quote:
User-agent: *
Disallow: :69696/
Sitemap: http://siteexample.com
pirithous is offline   Reply With Quote
Old 3rd May 2014, 14:17   #24
thinktink
Forum King
 
thinktink's Avatar
 
Join Date: May 2009
Location: On the streets of Kings County, CA.
Posts: 3,007
Send a message via Skype™ to thinktink
The robots.txt specification does not work for sites on different ports. You would have to serve the robots.txt from the DNAS port. But, if you're using version SHOUTcast DNAS 2 RC (Build 27) 07/18/2011 or later of the DNAS then you don't have to worry about it as those already serve the correct robots.txt file. You can verify it by going to http://siteexample.com:69696/robots.txt



You should see something like the following:
Quote:
User-agent:*
Disallow:/
thinktink is offline   Reply With Quote
Old 4th May 2014, 06:37   #25
pirithous
Junior Member
 
Join Date: Feb 2014
Posts: 26
I added the robotstxtfile parameter to the DNAS config file, specifying the path to where the robots.txt file resides. One question, though, is that all the documentation I found pertaining to creating robots.txt files talks about excluding certain directories that obviously begin with a forward slash, and not beginning with a colon -- which is what I did as you can see above. Do you know if this will work or not?

An example of a standard robots.txt file is seen below:

Quote:
User-agent: *
Disallow: /cgi-bin/
Disallow: /tits/
Disallow: /junk/
pirithous is offline   Reply With Quote
Old 4th May 2014, 14:06   #26
thinktink
Forum King
 
thinktink's Avatar
 
Join Date: May 2009
Location: On the streets of Kings County, CA.
Posts: 3,007
Send a message via Skype™ to thinktink
Actually, the default robots.txt that the DNAS generates if you don't provide one blocks everything. You shouldn't need to specify one for this situation.

As far as I know, in a robots.txt file all resources start with the forward slash to indicate server path root, the first directory. Here's some light reading on the subject:
http://www.robotstxt.org/orig.html

I'm at work right now so replies will come in slow if you have more questions.
thinktink is offline   Reply With Quote
Old 5th May 2014, 15:06   #27
DJ-Garybaldy
Major Dude
 
DJ-Garybaldy's Avatar
 
Join Date: Sep 2003
Location: Harpurhey, Manchester UK
Posts: 1,001
I've actually found a great way of thwarting steamrippers

I use Altacast (A Fork of Edcast) to stream with and set the metadata to update on 35 second interval. That way the song title doesn't change when the song does. Don't seem to have a problem with them.

I know you will never stop them from trying but it's a start.



Proud USER of RadioDJ since 2010

Online: Twitter - Blog - RadioDJ
DJ-Garybaldy is offline   Reply With Quote
Old 5th May 2014, 16:37   #28
pirithous
Junior Member
 
Join Date: Feb 2014
Posts: 26
Quote:
Originally Posted by thinktink View Post
Actually, the default robots.txt that the DNAS generates if you don't provide one blocks everything. You shouldn't need to specify one for this situation.
How is it disallowing everything when it only contains what is seen below? Google had indexed our old port number, so it looks like that may not be the case.

Quote:
User-agent:*
Disallow:/
Quote:
Originally Posted by garybaldy72uk View Post
I've actually found a great way of thwarting steamrippers

I use Altacast (A Fork of Edcast) to stream with and set the metadata to update on 35 second interval. That way the song title doesn't change when the song does. Don't seem to have a problem with them.

I know you will never stop them from trying but it's a start.
Looks interesting, but we only use BSD and Linux for server purposes, so we would need something that works on either of those platforms. Windows has a number of disadvantages pertaining to inherent weaknesses that in my opinion, make it more susceptible to malware attacks and cracking. Security through obscurity has been debunked a long time ago, not to mention that user subjugating software frequently has backdoors and spy features.
pirithous is offline   Reply With Quote
Old 5th May 2014, 19:26   #29
DrO
 
Join Date: Sep 2003
Posts: 27,873
just a / means ignore everything on the DNAS (since it's the base path of everything). though there's other ways which your stream could have been indexed (or it was done before using the newer DNAS since it's only present with some of the v2 DNAS builds and definitely not present with the v1.x DNAS).
DrO is offline   Reply With Quote
Old 6th May 2014, 12:57   #30
DJ-Garybaldy
Major Dude
 
DJ-Garybaldy's Avatar
 
Join Date: Sep 2003
Location: Harpurhey, Manchester UK
Posts: 1,001
Quote:
Originally Posted by pirithous View Post

Looks interesting, but we only use BSD and Linux for server purposes, so we would need something that works on either of those platforms. Windows has a number of disadvantages pertaining to inherent weaknesses that in my opinion, make it more susceptible to malware attacks and cracking. Security through obscurity has been debunked a long time ago, not to mention that user subjugating software frequently has backdoors and spy features.
I run my copy of Altacast on Windows 7 in a Virtual box environment on Ubuntu studio.

None of the programs I use for streaming with have Spyware in them they are just full working free programs that do the job.

I know of several shoutcast streaming programs that do report back to the company that owns them such as SAM with it's reg key system. Even simplecast did the same thing.

I won't allow a program near my system if i think it's got spyware or other nasty malware in it. Learnt my lesson back in the early days of XP!



Proud USER of RadioDJ since 2010

Online: Twitter - Blog - RadioDJ
DJ-Garybaldy is offline   Reply With Quote
Old 6th May 2014, 18:01   #31
pirithous
Junior Member
 
Join Date: Feb 2014
Posts: 26
Thank you to thinktink and DrO for your help. Everything is up and running, and the stream ripping has stopped.

Quote:
Originally Posted by garybaldy72uk View Post
None of the programs I use for streaming with have Spyware in them they are just full working free programs that do the job.
How can you be sure if you can't view the source code? Do you mean "free", as in free beer?

Quote:
Originally Posted by garybaldy72uk View Post
I know of several shoutcast streaming programs that do report back to the company that owns them such as SAM with it's reg key system. Even simplecast did the same thing.
I'll take your word for it, but I don't think older versions of SAM phoned home. They did, however, hide multiple keys in the Windows registry which didn't get uninstalled upon uninstalling the program. That's how it knew when the trial expired. These days, it looks like most consumers are mainly interested in getting things through app stores, which means the Windows registry will most likely become deprecated at some point. Even though somebody like me is against a feudalistic and totalitarian app store that includes a remote kill switch in it (back door), I can see how consumers want the convenience. Gone will be the days of finagling with .msi files, cleaning up a bloated registry, and dealing with a system which slows down as time goes on. Maybe someday, Windows will just work. Me, on the other hand, will stick with Linux and BSD. Either the program controls the users, or the users control the program. I prefer the latter.
pirithous is offline   Reply With Quote
Old 6th May 2014, 19:07   #32
DJ-Garybaldy
Major Dude
 
DJ-Garybaldy's Avatar
 
Join Date: Sep 2003
Location: Harpurhey, Manchester UK
Posts: 1,001
How can i be sure I know the FREE programs I use don't have spyware in them? That's an easy question to answer.

I work closely with the developers of both Altacast and RadioDJ neither have spyware or any malicious code in them. That's how I know what's what!



Proud USER of RadioDJ since 2010

Online: Twitter - Blog - RadioDJ
DJ-Garybaldy is offline   Reply With Quote
Old 6th May 2014, 19:17   #33
pirithous
Junior Member
 
Join Date: Feb 2014
Posts: 26
Quote:
Originally Posted by garybaldy72uk View Post
How can i be sure I know the FREE programs I use don't have spyware in them? That's an easy question to answer.

I work closely with the developers of both Altacast and RadioDJ neither have spyware or any malicious code in them. That's how I know what's what!
But did you review the source code? Is the code available for me to review? It doesn't look like it. Even if there weren't any intentional spy features or back doors in the code, how do you know there wasn't an unintentional one put in the code? You don't, so your argument doesn't work -- unless you personally reviewed the source line by line (and are proficient in whatever language it was written in). And even then, people miss stuff, which is why the source code of open source software needs constant review by as many people as possible.
pirithous is offline   Reply With Quote
Old 13th June 2014, 18:12   #34
DrDownhill
Junior Member
 
Join Date: Jun 2014
Posts: 4
MusicDNA is the StreamRipper

hi, i too have the same issue with a blank field listener. after i tracert the ip (209.239.114.980) it goes to a company called 'musicdna-interface'

after looking up via search what 'musicdna' is i learn the are a for-profit commercial entity that, amongst other things, acts as an investigator and estimator of royalty payments.
http://www.musicdna.com/en/our-service/index.html

musicdna is ripping your broadcast and creating a library of music for a data base of monetization forecasting for perceived royalties. in my case, they are not listening to the broadcast as intended, but rather ripping my stream for their self promotion and profit.

does anyone else have thoughts and/or opinions about this?
DrDownhill is offline   Reply With Quote
Old 13th June 2014, 18:17   #35
DrDownhill
Junior Member
 
Join Date: Jun 2014
Posts: 4
MusicDNA breaking though my ban list

oddly, i banned the musicdna ip and subnet earlier this week and found they were back earlier today ripping my stream (for profit) and that they were somehow 'magically' off of my ban list.

i just re-banned their ip and subnet now, but has anyone else experienced a user breaking into their shoutcast admin portal and altering the ban list to re-enable their ip?
DrDownhill is offline   Reply With Quote
Old 13th June 2014, 19:35   #36
DJ-Garybaldy
Major Dude
 
DJ-Garybaldy's Avatar
 
Join Date: Sep 2003
Location: Harpurhey, Manchester UK
Posts: 1,001
Quote:
Originally Posted by DrDownhill View Post
hi, i too have the same issue with a blank field listener. after i tracert the ip (209.239.114.980) it goes to a company called 'musicdna-interface'

Are you sure that IP is correct there is no .980 IP ranges only go from 0-255



Proud USER of RadioDJ since 2010

Online: Twitter - Blog - RadioDJ
DJ-Garybaldy is offline   Reply With Quote
Old 16th June 2014, 15:49   #37
DrDownhill
Junior Member
 
Join Date: Jun 2014
Posts: 4
good catch - ip is 209.239.114.98

Quote:
Originally Posted by garybaldy72uk View Post
Are you sure that IP is correct there is no .980 IP ranges only go from 0-255

i must have put an extra '0' a the end.

AND THE MOFO HAS GOT THROUGH MY BAN LIST AGAIN!
HE IS BACK TODAY!
DrDownhill is offline   Reply With Quote
Old 16th June 2014, 15:52   #38
DrDownhill
Junior Member
 
Join Date: Jun 2014
Posts: 4
so the musicdna-interface ip of 209.239.114.98 has breached my ipbanlist *AGAIN* and is back streamripping my music broadcast for their commercial gain.

any ideas how:
a) they are 'unbanning' their ip from from my ban list?
b) how to really ban them from my ban list?
DrDownhill is offline   Reply With Quote
Old 16th June 2014, 18:46   #39
DrO
 
Join Date: Sep 2003
Posts: 27,873
without knowing the DNAS version you're using, had to say.
DrO is offline   Reply With Quote
Old 18th June 2014, 14:26   #40
DJ-Garybaldy
Major Dude
 
DJ-Garybaldy's Avatar
 
Join Date: Sep 2003
Location: Harpurhey, Manchester UK
Posts: 1,001
Quote:
Originally Posted by DrDownhill View Post
so the musicdna-interface ip of 209.239.114.98 has breached my ipbanlist *AGAIN* and is back streamripping my music broadcast for their commercial gain.

any ideas how:
a) they are 'unbanning' their ip from from my ban list?
b) how to really ban them from my ban list?
Daft question you did choose a different admin password than the default "Changeme" didn't you?? That's the only way someone would be able to get into your admin and unban themselves.



Proud USER of RadioDJ since 2010

Online: Twitter - Blog - RadioDJ
DJ-Garybaldy is offline   Reply With Quote
Reply
Go Back   Winamp & Shoutcast Forums > Shoutcast > Shoutcast Discussions

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump