New! Search by Melody

[pml]

We have a new item on the front page, which I’d love to be able to tell you about... But Safari on iPhone doesn’t support the required level of JavaScript... (or otherwise Feldmahler’s still fiddling about!)

Cheers, PML

EDITED TO ADD:

It works quite well, presenting you in the top left-hand corner with a representation of a two octave keyboard. Melodies are normalised to start on A (so if you feed it something with a starting note of E flat, you disconcertingly see it transposed up an augmented fourth on the read-out to the right, but that’s not really a problem).

Once you’ve added your melody and clicked the search button, you’re presented with a time graph* with an oscilloscope-like tracing of search results over that continuum, and some links to narrow in on a given time band. So say you enter Eb2 G2 Eb2 Bb1 Eb2 G2 Bb2 Eb2 and click search, then you find a big spike around the year 1803, with a number of much smaller spikes all the way back to 1698 and forward to 2008. The result for 1803 links to about 7 different IMSLP scores by page number (as well as providing the obligatory EweToob link).

It knows the repertoire quite well. I fed it a few peculiar examples designed to trick it, and surprisingly it didn’t always fail. Angular melodies are much more easily detected than ones that move by step; it took about three or four different melodies from a piece by Schönberg before it could grab the correct piece. Beethoven’s 5th is easily found (though the last movement theme also picks up some Mozart with similarities!) The L’homme armé tune (either the major or minor version) fails dreadfully. It even knows some of my own pieces**. But it didn’t identify the piece of Bach in a recent discussion when given the first 9 notes (which have an ambit of an octave).

If I have a criticism, it is that the scaling of the time graph is often too narrow or wide for the number of results, and there doesn’t seem to be a way to alter that. Based on the melodic search, you’re given several time intervals to investigate, and some are too narrow while others are too wide; as the results are date sorted, if the search picks up a huge number of results and you know the one you want is almost at the end of the interval, then you have to click through screen after screen of “Next” results. Especially as the time graph clearly ranks some results much higher than others, it would be preferable to have the first screen report the highest-ranking results, and subsequent screens could then list the remainder chronologically.

Executive summary: a very useful tool!

* Going all the way back to 1198… I wonder if my datings for Pérotin’s conductus pieces Viderunt omnes and Sederunt might have a bearing on that? (Yes!)
** I fed it motifs from Opp. 1 and 5 which it successfully found.

[imslp]

Nice review PML! Just wanted to note that the reason it does not work on iPhone/iPad is probably because Flash is required for the piano keyboard and also the graph.

[pml]

I suspected there were some dark arts involved. Flash... eww.

ETA: And I discovered that when you’re focussed in on a time interval, you can pull the graph to the left or right to change the focus of the time segment. But if you’re hovering over the year 1877 and it tells you there are 35 hits or so many % under the curve when smoothed out, you can’t click directly on it to display the results for 1877, which is a bit maddening.

The worst aspect of this was being given a search interval from 1601 to 1724, and the two best-ranked results were dated 1713 and 1720 respectively. Had to click “Next” about a dozen times to get all the way through to about 1701, only for the applet to reach its limit of “Next” clicks, so that no results after 1701 were shown!

[viro]

PML,

thanks for a nice review, I appreciate the feedback!
I've just added the possibility to narrow the time range for the score search using the chart.

Vladimir

[andrewt]

Awesome! How does it work??? The only way I can think of would be to OCR all of the scores...? Does it do any part, or just melody?

[viro]

Andrew, indeed, I just checked, I started OMR'ing the scores one year ago, to the day!
The index contains all voices, so no, it's not only the melody.

Currently only the simplest index is online - there all voices are indexed separately and are linearized, meaning that a chord is represented as a rising sequence of its notes. So sometimes it's not trivial to find the place in the score from the search results which corresponds to the entered melody - the melody might be hidden inside the chords!

But this is of course just the beginning. I will build more indexes so that one can search by other meaningful entities, such as chords, rhythms, their combinations, etc. Stay tuned! (e.g. via @peachnote on Twitter)

Vladimir

[viro]

PML,

now for time intervals containing less than 2048 indexed scores for a given melody, the scores are sorted by the number of pages the melody occurs on. It's not quite the same as the number of occurrences (which is depicted on the graph), since pages containing many occurrences of the melody are treated the same as pages where the melody was found only once, but I guess this is a minor issue. If people will request the ability to change the ordering (from page count to occurrence count), I'll implement that.

Vladimir

[pml]

I think you’ve done a terrific job programming this Vladimir.

I noticed one apparent change in the behaviour of the applet earlier, which was that changing the focus of the time segment automatically updated the list of scores to the right; maybe Flash had buggered up on my system when I made my first comments upthread. For some melodies, especially through the late 19th century (the best represented era on IMSLP) you get very large numbers of results, and if some works score better than others as shown by the graph, it would help have the option to display the highest-ranked results first, in addition to chronological sorting. I can see value in wanting to parse the results both chronologically for some searches, and by rank/level of matches for other searches.

I also discovered by means of making a note entry mistake that you can have two different melodies compared (using red and blue lines for their graphs). The keyboard interface isn't very tolerant of mistakes. After seeing how it processes a melody the formatting it generates is pretty obvious (it counts semitones). So say I hit the mouse at the wrong moment and get a G instead of A; okay, I can delete the last number and begin clicking notes again — or I can add 2 to the number that erroneously has been entered as the most recent melodic step (to get to A instead of G) and resume adding notes. In practice though I can’t get it to resume adding notes to the current melody after I’ve corrected a mistake — the app treats the resumption as a new melody rather than the continuation. So usually it is finicky enough that the whole input sequence has to be deleted and played in again from the start. It’s a little bit annoying, but as we’re only dealing with 11-note phrases maximum it’s not a huge aggravation!

Oh, and I love the “I’m feeling lucky” button that you obtain for a null result search (along with the polite apologetic). That’s a well-thought out feature! ;-)

Cheers, Philip

[viro]

Philip,

thanks again for you feedback! I fixed that issue with the piano keyboard typos.
Regarding the ordering of search results, I guess that for very short / very frequent melodies there is not much point of finding scores containing most of them. These are more interesting for research purposes.

For example, check out the following graph: http://www.peachnote.com/#!nt=singleNot ... +14,57+-14

Here you see how different intervals have been used throughout the time. You can notice that at the beginning of the nineteenth century the rising and falling intervals diverged, with rising being more frequent for small intervals and falling for intervals larger than an octave.
Some, like the quint (http://www.peachnote.com/#!nt=singleNot ... 57+7,57+-7), are direction invariant, while others, like the quart, (http://www.peachnote.com/#!nt=singleNot ... 57+5,57+-5), have always been used more in the rising variation.

And for relatively unique melodies, like for example the Ode to Joy motif (http://www.peachnote.com/#!nt=singleNot ... %20-2%20-2), the search results are now sorted as expected. Currently is't by the number of pages containing the melody (while the chart shows the total number of occurrences, counting multiple occurrences on the same page). But the results seem to stay reasonable, with the Ninth being the first result

PS: B-A-C-H - http://www.peachnote.com/#!nt=singleNot ... 58+-1+3+-1

Cheers,
Vladimir

[viro]

Regarding the "I'm feeling lucky" button, as nice as it was, it's gone for now. Instead, you simply always get a result when searching for a melody. The found melody might be just a bit shorter than the entered one.

Also, for queries with less than 100 search results, they are shown immediately, without the need of selecting the time period first.

What do you think of these changes?

[aldona]

OK, I gave it a Schubert test...fed in the first and second themes from the Unfinished Symphony, the start of the "Great" C major symphony, and the start of the Arpeggione Sonata...and it did not recognise a single one of them. (and yes, I did try in the original keys and transposed to start on A.)

Maybe it doesn't quite know the WHOLE repertoire yet...

AJ

[imslp]

I think Vladimir was right when he asked me to change the "(New!)" label beside the feature to "(beta)". The project is indeed very much a work in progress, and not even all of the files are processed yet.

Also, I've found that I have much more success feeding it the first few notes (e.g. 8-10 or so) of the melody instead of the entire melody. When I don't find something, I just keep removing the last note. Usually the search ends up finding something in the end, in many cases the correct work.

Vladimir: Perhaps it might be interesting to make the search automatically truncate the end of the melody if it can't find anything?

[Mostad]

Wow! That's a useful feature!

Just one minor issue: I am wondering about the choice to limit the octave to 12 notes and not include non-enharmonic notes. Most music on IMSLP would not fit into this scheme since it is more than 70 years old. For instance: If I search for BACH and the keyboard on the screen and search engine does not separate A# from Bb, but rather treats them equally, I would also get music with A#, A, C, H, wich are not nessecarily connected with Bachs name at all. A# is a lower pitch than Bb for many of us. When searching for pieces with a major third, for instance the third in E flat major (Eb-G), I would also get pieces with a diminished fourth not being in E flat major at all (D#-G), even if I didn't want to serch for pieces with diminished fourths. If I wanted to find diminished fourths, I would also get major thirds, and I would have to go through vast amounts of music to find what I was actually looking for.

If the point of the search is to be able to search from melodic interval structure, then there is a great deal of difference between an augmented secound and a minor third &c (unless we are searching for dodecaphonic music in equal temperament). Why not use a keyboard with split black keys and give us the possibility to choose between flats and sharps? The octave has got at least 17 notes...

I know this question might seem a bit pedantic, but as the examples hopefully conveyed, it does make a difference. Even now, enharmonic notes are usually differentiated in notation of tonal music, because the logic of harmony demands a b flat for a g minor chord and an a sharp for an F sharp major chord since traditional western harmony is based on staples of thirds. If you do not believe me about equal temperament just recently becoming the most common temperament, then read Paul Tortelliers cello method, listen to some early 20th century recordings and read Strauss' edition of Berlioz' book on orchestration. Of course, equal temperament was used by some people in the past, but it was not the most common temperament untill recently.

Otherwise I found this search engine extremly impressive!

[pml]

The octave has 35 notes if you allow every enharmonic possibility with naturals, sharps, flats, double sharps and double flats (and higher degree accidentals are theoretically allowable in very remote keys). But the tonal system since 1800 has usually assumed closure and enharmonic equivalence, so limiting to twelve note pitches is completely in keeping with that. It's a pity the melody doesn't allow searches on scale degree – so that you can search on a pattern where a certain note in the pattern is locked in as "i" or "v"; or that the spelling of a progression has to be diatonic rather than chromatic (i.e. B flat — A rather than A sharp — A): but these are minor criticisms.

Cheers, Philip

[lexluthorxalo]

"Search by melody" don't have "note value (http://en.wikipedia.org/wiki/Note_value)"...???..