milkdrop changes in 5.57 beta

[Benski]

hey guys, updated to ns-eel2 in the version of milkdrop in 5.57 beta. should support all the same functions as AVS now (except for getspec/getosc). even megabuf/gmegabuf should work but no guarantees on them working successfully or even being useful

also, this should remove the 8 character variable limit.

let me know if you guys find any issues

[ravermeister]

nice to see that milkdrop is still under an eye of an developer.
why isn't it opensource anymore, or open in anyway you can
retreive updates from the community? I think that would bring
MD much faster forward...

[Koopa]

I doubt it would speed up development.

Many people asked for an open source MilkDrop/AVS in the past, but after the source code was released, there was not much.

Only RediJedi made some tweaks, projectM could fix some bugs in their project with the source code and this it was.

I haven't seen any custom AVS build.

It seems, that all these people, who originally asked for the source code are no longer on the planet, hehe.

Also don't forget that MD 2 was a major upgrade, doubtful that AOL would pay money for it and then release the source code.

MilkDrop2 is one of the main advantages of Winamp.

And yes, thanks for working on it Benski.

[Nitorami]

Maybe it's a stupid question but what is ns-eel2 and what does this actually mean ???

[Koopa]

It's a library that powers AVS and now MilkDrop too.

http://www.cockos.com/wdl/

Hopefully Benski will give you a better explanation.

[QOAL]

AVS is a legal minefield licensing wise, so it's best for any non-AOL developers to stay away from it, even Justin has mentioned that himself. (That and as I'm sure all people that have touched AVS know, it's buggy and hard to compile correctly.)
Fridge is being developed by some of the finest AVSers the scene had, which will use the GPU to render as opposed to the way AVS does it - It's open source so if you want to help develop that then you can. (Look in the AVS sub forum)

EEL2 is, unless I'm mistaken, actually better than the EEL in AVS. (So curse you MilkDrop )

What it does is it takes the users code input and compiles the code on the fly so it can act natively within the program (Well within a VM). Defining any new variables as it goes.
Without it you'd find it very limiting to produce any presets since you'd be stuck using predefined variable names and you wouldn't be able to do anything fancy with maths.
Blah = 1, instead of blah = sqrt(woo * cake)
Pretty much. (Shout at me if I'm wrong though)

[Flexi]

Originally posted by Benski
hey guys, updated to ns-eel2 in the version of milkdrop in 5.57 beta. should support all the same functions as AVS now (except for getspec/getosc). even megabuf/gmegabuf should work but no guarantees on them working successfully or even being useful

megabuf (in combination with loops) is working fine!
But i wonder what other new functions are now added to the Milkdrop script language.
The AVS tutorials helped a lot, but i don't want to miss any new features.
I've searched but I didn't find any function reference.
If someone points me to an overview of the ns-eel2 functions I could also update the Milkdrop readme.

[Flexi]

Originally posted by Benski

let me know if you guys find any issues

it seems that some shader presets now compile to a higher shader operation count so that they don't fit the 64op limit for shader model 2.0 and they throw error messages (when not raising the preset's shader model level).
For example the 'flexi + fishbrain - operation fatcap 1' now needs 65 operations... any ideas?

[Benski]

Originally posted by Flexi
megabuf (in combination with loops) is working fine!
But i wonder what other new functions are now added to the Milkdrop script language.
The AVS tutorials helped a lot, but i don't want to miss any new features.
I've searched but I didn't find any function reference.
If someone points me to an overview of the ns-eel2 functions I could also update the Milkdrop readme.

Here's the list in MD2.2 The ones that start with underscore are just internal equivalents to syntactical stuff like / or *

• if
  band
  bor
  loop
  while
  bnot
  equal
  _noteq
  below
  above
  _beleq
  _aboeq
  assign
  _mod
  _mulop
  _divop
  _orop
  _andop
  _addop
  _subop
  _modop
  sin
  cos
  tan
  asin
  acos
  atan
  atan2
  sqr
  sqrt
  pow
  sigmoid
  _powop
  exp
  log
  log10
  abs
  min
  max
  sign
  rand
  floor(Sameasint)
  int
  ceil
  invsqrt
  megabuf
  gmegabuf
  freembuf
  memcpy
  memset

[Benski]

Originally posted by Flexi
it seems that some shader presets now compile to a higher shader operation count so that they don't fit the 64op limit for shader model 2.0 and they throw error messages (when not raising the preset's shader model level).
For example the 'flexi + fishbrain - operation fatcap 1' now needs 65 operations... any ideas?

MD2.2 is using a newer version of the D3DX libraries, which includes a newer compiler. I had to turn on compatibility mode or a lot of the presets would break. I'd imagine that the compiler is doing slightly different things that results in a different number of instructions. Probably has to do with the optimizer.

[Benski]

NS-EEL2 is the equation evaluation library that powers MilkDrop 1.x and the first stages of the pipeline in MD2.

I made a heroic effort to port AVS to use EEL2, but the APEs (vis_avs plugins) don't work unless vis_avs is compiled with an ancient version of the microsoft compiler. It uses a C++ interface and unfortunately the binary layout of C++ objects can (and does) vary between compilers, even different versions of the same product. If I can get EEL2 working under the older compiler (it might ...) then perhaps there will be an upgrade. EEL2 won't add much to AVS but it might be slightly faster.

As for open-sourcing MilkDrop 2, there's certainly a number of issues. As Koopa said, the lack of enthusiasm around AVS and MilkDrop 1 is very disappointing. Also, it didn't take very long for these visualizers to show up in competing products. Although having community help on the visualizers would be great as I have only 24 hours in a day and have to sleep for at least a few of them I might be able to work something out but no promises.

Originally posted by QOAL
AVS is a legal minefield licensing wise, so it's best for any non-AOL developers to stay away from it, even Justin has mentioned that himself. (That and as I'm sure all people that have touched AVS know, it's buggy and hard to compile correctly.)
Fridge is being developed by some of the finest AVSers the scene had, which will use the GPU to render as opposed to the way AVS does it - It's open source so if you want to help develop that then you can. (Look in the AVS sub forum)

EEL2 is, unless I'm mistaken, actually better than the EEL in AVS. (So curse you MilkDrop )

What it does is it takes the users code input and compiles the code on the fly so it can act natively within the program (Well within a VM). Defining any new variables as it goes.
Without it you'd find it very limiting to produce any presets since you'd be stuck using predefined variable names and you wouldn't be able to do anything fancy with maths.
Blah = 1, instead of blah = sqrt(woo * cake)
Pretty much. (Shout at me if I'm wrong though)

[InCUbuS-94]

and where can i download this new version?
it's inculded with new winamp 5.56?

[DrO]

it's in the 5.57 beta - see thread in main winamp discussion forum.

-daz

[rce12]

i can download it on filehippo.com, but i can't see what this newest milkdrop is

[DrO]

you really shouldn't be getting the beta version from anywhere other than via the official download site from here.

-daz

[Flexi]

i don't see the exec2 and exec3 functions which are essential for loops in that list, though they are working.

and how do you use memcpy and memset?

[Stupifier]

I'm not really in the know on MilkDrop preset development....I only use the presets others post on here and they are amazing.

Frankly, I just wanna know if this update means there are going to be some dope ass new presets coming out soon due to new capabilities? Any preset makers wanna answer that for me?

Thanks.

[Flexi]

The greatest change so far is the megabuf support. The megabuf is an Array to store a million values.
Before the update Milkdrop preset authors were limited to maybe 40 variables. As for me, i have maxed this limit out several times. (the elastic rope is intensive on variables use, and the bouncing objects too).

Milkdrop is still the same, but the update truly allows to use a whole new bunch of algorithms. Not to imagine what presets will there be in a year, it's really hard to predict anything else but the abstract term 'complexity'.

wait and see (or even better: try and find out)

[Stupifier]

I see, thank you for the clear explanation. Sounds like some cool stuff is happening.

[ShadowHarlequin]

is there any updated documentation/wiki pages with all the new stuff added? Flexi mentioned the avs tutorials as being useful, any idea which ones?

[Nitorami]

Finally got some time to play with the new MD features as of 5.57 beta. Found no problems so far, except for the nightmare syntax, which forces me to nest confusing exec blocks to buggery, where I just wanted to insert a single additional statement... anyway.

I thought now where we can store a million instead of just 32 variables, we can buffer the past few seconds of the volume, and exploit it for a better beat detection, using autocorrelation or Fourier transform. The attached preset demonstrates these experiments. What it does (in the frame equations):

1. Create an average free volume variable (vol). I used bass+treb, which probably contain most of the rhythm information, and subtracted its lowpass filtered value.

2. Push it into a ring buffer. I chose a buffer size of 120, sufficient to store 4 seconds of vol data (at 30 fps). On each frame, the whole buffer content is moved one place, as in a shift register, i.e.
buf(120) will be lost
buf(119) will be moved to place 120
buf(118) will be moved to place 119
etc.
The new vol value is written to buf(1). Now we have a history of the last 4 seconds vol data. This is shown in the moving pink curve (newest vol value on the left hand border).

3. Perform an autocorrelation on this buffer. Nothing mysterious and actually quite simple to calculate. Correlation is generally a measure for the similarity between two blocks of data. In our case, we calculate the correlations between the newest block of vol data, say the past 2 seconds, and older data blocks, in increments of one buffer place (1/30 sec):

corr(0) = corr (data(1..60), data (1..60))
corr(1) = corr (data(1..60), data (2..61))
corr(2) = corr (data(1..60), data (3..62))
etc.
corr(59)= corr (data(1..60), data (60..120))

Of course corr(0) will yield the highest value because it correlates between the same block of data. We can give it a miss. As the distance between the blocks becomes larger, the correlation will decrease, except if there is a repetitive pattern in the vol data. The autocorrelation reveals this pattern. For instance Gigi D'Agostinos "Another way" has a tempo of 129 BPM (beats per minute), one beat every 0.464 seconds, which at 30 fps corresponds to 14 frames, so we may expect the first correlation maximum for corr(14).
The green curve shows the autocorrelation, starting with 0 at the left hand border. The scale is 0.5 seconds per division. When playing "Another way", you'll note the first maximum at approximately 0.46 seconds as expected, the next at 2*0.46 seconds etc. Note the scaling is only correct if the preset is run at exactly 30fps !!!

4. The bottom two curves show the Fourier transform of:
- the autocorelated vol data (white)
- the vol variable directly (brown)
I scaled it in BPM: One division = 50 BPM. For "Another way", the first maximum appears, as expected, around 130 BPM (again, only if run at 30 fps).

As could be expected, Fourier or autocorrelation analyses work quite well with reasonably rhythmic pieces, but fail where the rhythm is not clearly expressed in volume. This is the same in principle as for a primitive bass based beat detection.
Additionally, the current simple implementation entirely depends on a constant fps rate, and goes wrong when fps changes. This problem can probably be overcome by a more sophisticated code but in general I think these schemes will hardly provide any significant benefit for beat detection.

[Nitorami]

I found that gmegabuf introduces a feature which might be quite useful: Since it can be read and written from all frame/wave/shape code sections, it allows the transfer of data "backwards", e.g. from the waves to the frames equations section, which was not previously possible via the q variables.

I used this to create a "true" volume variable. The standard bass, mid and treb variables are subject to automatic gain control (AGC) and do not reflect the real music volume. I often found this a pain because the preset never knows when there is a silent passage in the music. Luckily, AGC is not applied to the variables value1, value2 in the wave sections. Therefore I used wave 1 to generate a "true" volume by adding up all spectral components, and stored it in gmegabuf(1), which is accessible from the frames equations.

Attached a simple demo. The pink curve shows the traditional volume (bass+mid+treb, smoothed), the green curve is gmegabuf(1).

Play an average piece and both curves will be similar.

Then play a quiet piece and see how the pink curve slowly grows as AGC control opens up, as it always tries to adjust to a medium level. This has benefits of course but it means that the relation to loudness is lost. After a silence of a few seconds, the signal may even freak out completely. When a loud passage follows, the pink signal will shoot over until AGC control brings it back to average level.

Altogether you'll see that the green curve reflects the real volume much better. Similarly, you may now generate your own bass or treb, or other bandfiltered variables as you like, or generate separate variables for the left and right channel, simply by selection of spectral components in the wave equation.

[ShadowHarlequin]

someone posted a link to an avs guide about the new stuff available to milkdrop but i cant find it anywhere now, anyone have any idea where it is?

[Flexi]

have a look at the AVS guide by PAK-9: http://forums.winamp.com/showthread....hreadid=209060

I've found all that i needed for using loops and the megabuf in it!

[Nitorami]

I had planned to write a brief tutorial about the new features a few weeks ago, to spare others the hassle of reading through the long tutorial, but I never got round to it. When I finally started (an hour ago), I noted to my considerable surprise that most of what I could tell you (and what I learned from the AVS tutorial), is obsolete.

With one of the latest betas, they seem to have changed the compiler discreetly, making things much easier than before. For instance I found there is no need for assign() statements anymore, and also the cumbersome exec statements have gone. Wonder why nobody told us. Maybe it's a bug, not a feature ? But definitely my code now works without assign and exec2/exec3... which is a huge relief I can tell you. Hope that will be official.

Ok, now to the features... can keep it much shorter than planned.

Global variables
These are read- and writable from anywhere except shader code.
gmegabuf() array can store 1 million values***
reg00 ... reg99 can store 99 values, similar to q1 ... q32 (except that regxx and gmegabuf work in both directions, e.g. can be written in the wave section and read in the frames section, while the q vars are unidirectional only)

Local variables
megabuf() array can store 1 million values but these are only valid locally, e.g. within the section and cannot be used to transfer data from frame to waves etc.

Loops
Array of millions of values would be useless without a way to fill them. Therefore we have loops to do that.

loop (n, statement1; statement2; statement3; etc. );
n is the number of times the loop is repeated, statements are normal lines of milkdrop code. As far as I tried, you can add as many statements as you like, separated by semicolons.

And that's it folks, unless Benski introduced other secrets I haven't discovered yet.

***The tutorial says that megabuf/gmegabuf can store 1 million values, although I could successfully address gmegabuf (20000000); - 20 millions. Not sure whether this is intended or rather a lack of range checking... suggest you are careful and don't use more than a million. Should be enough for the moment.

Martin

[Nitorami]

Hi all

Looks like the ns-eel update is quite a construction site... I think Benski owes us a few explanations soon

Just unveiled (by experimenting): conditional loops

while (condition; statement1; statement2; (...) ; ) ;

will execute all statements as long as condition is not zero.

Example - the following code calculates the faculty (result = count!).

result = 1;
count = 4;
while (above(count,0);
result = result * count;
count = count-1;
);

In principle conditional loops can be cobbled together by a normal loop of fixed length, encapsulating all statements by indiviual if() conditions, but that is unelegant and significantly less efficient.

One obvious use could be a quicksort algorithm, which sorts a number of shapes by their z-coordinate, to have them drawn in the right order, so the shapes in the front would actually be drawn in front of those in the back.

[Nitorami]

Hmm, maybe I was too optimistic... either have misinterpreted the syntax or it is still buggy.

This code should result in index=5

index = 0;
while (below(index,5); index=index+1; );

but actually delivers 1; the while loop stops prematurely. For any other increment than 1, it will stop one cycle later than expected; e.g. for increment 20, index ends up at 40 instead of 20.


In this code

index = 0;
while (below(index,-5); index=index+1; );

the while loop would, to common understanding, not be executed at all but it actually runs once and also results in index = 1.
Better don't use while loops for now. Still hope they will be implemented in the final version.

[redi jedi]

well shoot... I may have dust off my winamp install and check it out..

good to see some of the same faces(read:usernames) around here...

oh and Benski: I'd make at least a few mods if you can get me the source(Open source or not, I'll sign an nda if I have to)

[redi jedi]

@Nitorami

your syntax is incorrect:
index = 0;
while (below(index,5); index=index+1; );

should be:

index = 0;
while (
exec2(
index=index+1,
below(index, 5)
)
);
monitor=index;

the exec2 function "executes" both arguments and returns the second one. the while function will re-run itself until it gets returned a 0 or it gets to 1048576(change the below(index, 5) to 1 to test yourself)
there is also a exec3 function


its looks like the "while" is really a do...while as it always executes the first time

[ShadowHarlequin]

mods? whats this about mods? anything we may be able to get hold of in the end?

[DrO]

he's probably refering to the ones made against the MD 1.x source code.

-daz

[Nitorami]

@redi jedi

Yes agreed, that's some sort of logic that seems to work.
BUT it brings back the blasted exec statements which I had hoped to get rid of.

I found that when I remove all exec and assign statements from my experimental codes, they worked fine starting with one of the latest betas... I would be really annoyed if the exec crap would now be re-introduced via a do loop.

[Flexi]

redi-jedi, yeah! I remember having played with your latest beta, haha . what changes would you want bring in again? I've been kidding around with a tiny reengineering lately and i was trying out some algorithms: all open source http://www.openprocessing.org/portal...iewBy=sketches

[redi jedi]

@Nitorami

I thought the exec statements where always there with that engine... ohz well either way I couldn't get it to work without them either, I find them quite annoying as well. I almost want to make a compiler that will make md code from something more like javascript, but if I don't get back into making presets I'm not sure that'll happen.. plus there isnt any good way to integrate that with MD anyway so you'd have to reload the preset after every edit.


@ShadowHarlequin && DrO

I'm saying if I can get my hands on the md2 source I'll make some mods to it, mostly I mean the ones dro is referring to that I made to 1.x, at least the ones that would still make sense, like pageUp/down in code view, and draw in background(something I miss alot), I think there were quite a few that would still fit, and I would like to think are missed... although I'm not sure that anyone that used my beta(I should have quit calling it that since it was solid, maybe then ryan woulda actually taken some of my changes) is still around

But I have a few other ideas as well, like shaders for each shape. what would really tickle my fancy is a public facing API to control stuff from a 3rd party app, like change/retrieve the preset, directory. send out custom messages, ect.. then I could make a preset editor that could tell md to reload the preset when you save, and won't kill your changes if you hit the wrong key

[redi jedi]

@flexi lol good times
looks like you got farther than I did with mine, what did you build it on?
I got it loading presets and running them, but I never finished the rendering loop, all I got was the custom waves and the warping. I used XNA and I couldn't figure out how to get it to load a shader from text. It ran on both the xbox and the computer though so that was cool, it wouldn't run on my zune though cause I used lua as the language because you can pretty much feed it straight MD code and it works, then you've got extra features for the presets.

[Flexi]

For the moment it's written in Java and i used the Processing Framework. With a resolution greater than 300x300 the framerate drops dramatically . But i plan to go for an OpenCL implementation.(see this great video series) If possible I want to keep up with Java.

As for Javascript, Greg (ShadowHarlequin) gave me this interesting link last night: http://blog.nihilogic.dk/2009/03/jav...-milkdrop.html
pretty cool, huh?

[ShadowHarlequin]

i watched a video from google io talking about 'canvas' and javascript it and can use openGL, wouldnt that mean better fps and warp/comp shaders?

[Nitorami]

Not as such. You won't easily find any quite so simple interface to shader programming as offered by Milkdrop. MD provides a simple link between audio ouptut and shader input in the first place. If you want to implement this on a different platform, you'll have to spend months of programming to establish this link in the first place, before you can even start to think about shader programming.
Then you will have to run your code through compilers etc. before you can see it... I don't think there is any other platform which allows you to just press Ctrl-Enter to see what you've just coded.

The construct of warp/comp shader has no particular relevance, this is just a simplistic and practical construction of MD.

The video speed (fps rate) does not depend on the platform, because all platforms will only pass your shader code to the driver provided with your video card for compilation. So does MD.

MD of course has restrictions. It is only a visualiser coupled to winamp, and in principle not comparable to a general platform such as OpenGL. Its benefit is extreme simplicity. If you feel that MD is not enough for producing visuals, go ahead and try with an open platform, and we'll talk again in three years time.

Martin

[Flexi]

months of coding is maybe a little bit exaggerated. but three years is a long time to produce something awesome!

[DrO]

@redi jedi: you're probably best trying to contact benski about such things though if you can easily provide changes against some of the aspects (if possible) then i can merge the patch in (quite a bit of the MD2 code is the same as the MD1 as far as interaction, etc is involved). though trying to go through benski/official route is probably the best option as my tinkering with the plugin is generally only to do with localisation aspects (or so it's meant to go...)

-daz