Feature Request -- Digital Music Files (Typesets)

[pml]

My A$0.02...

IMSLP is a score library repository. The primary goal is usable sheet music, and the de facto standard is PDF. I would argue that anything that gets submitted in any other format, without a matching PDF, is unacceptable.

It is way more future proof than anything else mentioned thus far - I'm unconvinced that MusicXML is likely to be better than Lilypond in this regard.

The vaunted superiority of Lilypond outputting to PDF is negligible - virtually any GUI-based musical typesetting package now supports this, as for example on Mac OS X as Yagan points out this functionality is part of the operating system, and there are numerous PDF printer drivers on Windows offering the same: CutePDF, PrimoPDF, Win2PDF etc. etc. The Unix/Linux world already has huge libraries of utilities to generate PDFs from outputs of music engraving systems.

I'm sorry to pick on one comment of yours in particular Vivaldi, but I've yet to see proof that "Lilypond's computer engraving is the closest we've got to matching traditional hand engraving of music scores". I've seen a number of scores varying in quality from excellent to abysmal. Just because it's a excellent tool and a free tool (compared to excellent but expensive tools such as Sibelius) doesn't mean it will be wielded any better. I would say, "show me the money"... :)

I am happy of course if contributors wish to supply extra source files for typesets, such as .ly, .sib, .mus, .tex, .mup, etc. etc. etc. As several people have said, these are often useful (e.g. meeting a need fro transposition or re-arrangement). However, a source file without a PDF to go with it, is in my view useless, to the 95% of visitors to the website who won't have the ability to use that particular format.

I also have to apologise in advance for giving offence, if I record my strong disagreement with kcleung's opinion about having the "courage to typeset". Who do you think is going to re-typeset, e.g., a thousand pages of a Tchaikovsky ballet score or a Wagner opera? Even if you did, who is going to proof-read it? I have been typesetting music at an amateur standard, and occasionally to professional standard, for most of this decade, and I can say with certainty that my own work is mostly "amateur". Why?

Firstly, professional typesetters work to a house style and rules, which are set out in advance to limit the variation inherent in the way one engraver will lay out a piece compared to another. In computer typesetting the professional will have already planned out the number of pages, the systems per page, aggregation of staves, etc., all in advance. The amateur generally doesn't do this in any systematic fashion, as evidenced by the numerous typeset scores all over the internet which obviously use the "default settings" with minimal variation.

Secondly, publishers employ proof-readers to pick up the invariable errors that occur (and optionally, may subject fresh typesets to the trial of a rehearsal before going to a full print run). With amateur typesetting you are lucky if anyone actually reviews the score at all, ever, or finds the errors before it's too late (in context of performance), and frequently mistakes are not reported back to the "publisher" for future revisions.

Again to criticise my own editions: I have to do my own proof-reading, since although I can ask for help from people, I am not in a position to employ them; and I frequently have found that my (amateur) proof-readers miss mistakes that ought to be blindingly obvious... but without professional experience in the field it is easy to overlook errors in musical notation. The only exceptions to this experience were paid commissions for typesetting, or a special case where the body automatically hands over works to a different typesetter with experience in the repertoire for comment.

Sorry for the rant, I think it ended up being 2 dollars rather than 2 cents. As I said, that's my opinion... feel free to disagree, I don't mind...

PML

[kcleung]

IMSLP is a score library repository. The primary goal is usable sheet music, and the de facto standard is PDF. I would argue that anything that gets submitted in any other format, without a matching PDF, is unacceptable.

I totally agree. Any source code without a matching pdf is not acceptble since not everyone has the necessary compiler for that source code.

I am happy of course if contributors wish to supply extra source files for typesets, such as .ly, .sib, .mus, .tex, .mup, etc. etc. etc. As several people have said, these are often useful (e.g. meeting a need fro transposition or re-arrangement). However, a source file without a PDF to go with it, is in my view useless, to the 95% of visitors to the website who won't have the ability to use that particular format.

This is the primary reason why I suggest it is a good idea if you typeset a score, you should be allowed to submit the source along, so that it can be corrected, improved or transposed by others.

I also have to apologise in advance for giving offence, if I record my strong disagreement with kcleung's opinion about having the "courage to typeset". Who do you think is going to re-typeset, e.g., a thousand pages of a Tchaikovsky ballet score or a Wagner opera? Even if you did, who is going to proof-read it? I have been typesetting music at an amateur standard, and occasionally to professional standard, for most of this decade, and I can say with certainty that my own work is mostly "amateur". Why?

PML

Don't get me wrong.... If there is an existing score, of course why re-typeset it? However in orchestral music, in many case you can only get the full score, but not the parts. This is particularly true in Chinese symphonic works, which become out of print very quickly and not reprinted. You are damn lucky if you manage to source the full score. Many orchestras resort to hand-copy the orchestral parts from the full score. Yeah, amateur re-typesetting is prone to errors, but it is *much more accurate and efficient than hand-copying!

I believe most of you (include pml) would prefer lilypond re-typeset than hand-copying scores or making every orchestral player read full score and be very busy at page turning. Moreoever if you retypest, you can give the source code (and of course the pdf files) away and others can be benefitted. This is one of the special case where re-typesetting a printed work (in full score only, no parts available) is justified.

[madtom1999]

A visual score is of no use to the blind.
MusicXML is compact and can easily be turned into a whole variety of scores. A score cannot so easily be turned into MusicXML or Braille Music without humand intervention*.
PDF - or Pointless Document Format as it should be called (it does nothing it says on the tin that cant be got more easily elsewhere) merely promotes one companies products.
Do you want this site to preserve and exchange music or make it another bastion of one company?

While its not quite there yet we can help MusicXML develop - we cant do that with PDF.
Take internationilistation. You can store a song with MusicXML and have the lyrics in as many languages as you like and only view the one you want.
In PDF you would have to have the same score re-made for every language.

*I'd love to work on a an OCR score reader if anyone knows of a GPL project.

[Yagan Kiely]

There are only the very few users of IMSLP that would want to do that. The overwhelming majority just want to print/read on screen. Ergo, PDF (equivalent) is the best format.

MusicXML is all good a well, but it isn't a universal file that is viewable by all and easily printed - which is what IMSLP is used mostly for.

No sheet music site/business anywhere in the world shares notation files, they share music.

If you allow musicxml, you must allow all the notation formats. When that happens, it turns every page into a stew of so many different files that your head spins. It would ruin the page to have so many different confusing files. Especially for those that just want to print.

[pml]

Hi Mad Tom `99,

Indeed, a visual score is of no use to the blind. Some people here are perhaps misunderstanding what IMSLP is. It is a project to gather any conceivable written music in the public domain, as well as appropriately-licenced copyrighted works.

Believe it or not, the vast majority (as in, I would guess, 99.99%) of public domain music is currently not represented here. IMSLP is a repository for works brought in from "outside", as it were.

By definition, a MusicXML file is a retypeset. That means we require the permission of the typesetter in every case for it to be hosted here; we do not and cannot work on the assumption that we could just go to a site like Mutopia and grab any MusicXML file we might like.

The primary goal is not to occupy vast amounts of human-hours of the existing volunteers replicating existing work unnecessarily. Having to scan already published PD works is one of the least wasteful methods, especially when you have your workflow as well optimised as someone like daphnis. The time spent doing MusicXML encoding would be far greater, by orders of magnitude, not just in preparation, but in assuring the copyright status of the typesets are acceptable to be hosted here.

I am only being a little facetious when I suggest that perhaps you should be asking the thousands of music publishers out there why they aren't publishing Braille versions of everything they produce? The answer is there isn't actually a demonstrable need. I have worked with blind and sight-impaired musicians in my career, and in every case the MD of the ensemble has had to go to the trouble (sometimes quite lengthy) of ensuring Braille editions or oversize prints of the music could be prepared in good time. In other words, its an "on demand request", not a blanket assumption of need.

I think you would have to convince us why we should be concentrating efforts on MusicXML for the vanishingly small rewards it would offer. I suggest also you reread the second paragraph I wrote about "future proofing". MusicXML has gone through three standards in the last four years and yes, it is still in development. Until the standard matures, any music prepared now in MusicXML 2.0, (or was prepared in 1.0, or 1.1) is going to be less "stable" than anything in one of the existing PDF definitions.

And you should do a bit more research about PDF yourself. It is an open standard, ISO 32000-1:2008 (published 1 July this year), and if you don't want to use the Adobe product (Reader or Acrobat), there are numerous alternatives: Foxit, Preview, Sumatra PDF, Xpdf, Evince, Okular, KPDF, Drumlin PDF reader, PDFedit, Inkscape, Embedded General EG-DocViewer PDF, ePDFView... My post above mentioned the ubiquity of PDF drivers so that any GUI- or text-based music typesetting packages should support printing to PDF as well as exporting to MusicXML. Currently on my Mac however I am perfectly happy with the OS X.5 instantiation of Preview, and haven't installed anything by Adobe at all.

That doesn't appear to be the "bastion of one company" to me, but you may beg to differ. :)

And yes, there are a couple of open source, music OCR projects, though I can't remember the name of the main one at the moment...

Regards, Philip

[pml]

Don't get me wrong.... If there is an existing score, of course why re-typeset it? However in orchestral music, in many case you can only get the full score, but not the parts. This is particularly true in Chinese symphonic works, which become out of print very quickly and not reprinted. You are damn lucky if you manage to source the full score. Many orchestras resort to hand-copy the orchestral parts from the full score. Yeah, amateur re-typesetting is prone to errors, but it is *much more accurate and efficient than hand-copying!

I believe most of you (include pml) would prefer lilypond re-typeset than hand-copying scores or making every orchestral player read full score and be very busy at page turning. Moreoever if you retypest, you can give the source code (and of course the pdf files) away and others can be benefitted. This is one of the special case where re-typesetting a printed work (in full score only, no parts available) is justified.

There are other better alternatives than hand-copying, which are slightly less time intensive. A few years ago I was part of a "rescue mission" when a publisher that should have known better sent the wrong parts for the Monteverdi Vespers (1610) to a festival, and the librarian only realised the extent of the problem three days before the first orchestral rehearsal. There was no time to have new parts sent since the publisher confessed to not having the right parts even in the country. The solution for one movement was to mark up a full score with hand-written cues, photocopy it several times, and then assemble a team of "cutters" and "pasters" to go through, and reassemble them into individual parts.

It's not elegant, but if the players have mistakes in their "parts", at least they will be the exact same ones that the conductor will have to work out!

This same method is now easily done with PDF scans of full scores. I did a variation on this trick a matter of weeks ago with the Mozart Coronation Mass, since I didn't want my viola section to go home at the interval (Mozart's Salzburg masses were written for violins and celli/bassi but no violas). Admittedly I copied and pasted out of a PDF full score and then ran it through music OCR to get it into my notation package, but it was still much, much quicker than either hand-writing or typesetting from scratch (and then I could change the clef and problematic double/tre cordes, add rehearsal figures and cues etc). So again: why not PDF?

Regards, PML

[pml]

One last rant from me, and then I'll shut up for today...

I'm currently doing NaNoWriMo, so I am writing a lot of words besides the ones in this thread!

Yesterday, I went to see the first one of these machines installed in Australia, and had an out of print book (the English publication of Hypnerotomachia, by Francesco Colonna, London: Waterson, 1592) printed for me in a matter of about 10 minutes:

http://en.wikipedia.org/wiki/Espresso_Book_Machine

The cost of the book was the equivalent of a normal paperback, although there is only a limited range of books available at present, between $10 and $20. When the machines are in "full production mode" they will be able to print conceivably ANYTHING, and that will of course include musical scores. So you go to the store and they're out of the Haydn Symphony/Mozart Piano concerto/Verdi opera FS or VS/etc. etc. that you need? Get a copy printed on demand.

The machine uses two printers (one for the pages, one for the cover), a sorter/collator, rasp and gluer, binder, a kick-ass guillotine, and then your book drops out of the slot at the end, still warm.

Naturally, an electronic format is used for the e-books - care to guess which?

Regards, PML (back to my novel, I promise)

[madtom1999]

@pml - presumuably the typesetting permission problem would exist for ANY format so correcting errors would be a problem! Sensible licensing will solve that, and many other problems.

I'm not sure what this site is about any more - I thought it was to provide a central repository for all music? But not stuff thats already published- like the US Congress or the British library?

While I agree that music publishers do no publish Braille music or in MusicXML that is partly ignorance, and when it is not ignorance it through cost, or more likely protecting their market share.

Yagan suggested that to publish MusicXML would require publishing all formats. I think you'd find that if you publish the XML then those applications that cant read it would rapidly add that facility - indeed they would probably offer you a CGI program to do it on the fly.

If this site offered the original score (in PDF if so decided) and an option for people to upload their MusicXML transcriptions of the same it would be a different and more powerful beast.
Many music companies fear the web and fear MusicXML for the simple reason that they are becoming redundant. It is an easy thing to write a program that checks melody strings against other MusicXML tunes. That would be a fantastic feature for any musician (though it may set the Cats amongst the pigeons). If this site chooses to deny MusicXML then another site will have to be set up to do that and that will devalue this site hugely.

[pml]

Hi again Mad Tom,

Yes, the typesetting permission problem exists for ANY computer format. Correcting errors isn't such a problem, since we are only permitting Creative Commons licencing for typesets, which solves some of those issues fairly neatly.

However it's the problem of getting permission from the typesetter in the first place, since the Wiki mechanism allows anyone to upload music files, so that a contributor here is not necessarily entitled to give that permission. For example, the Werner Icking Music Archive has articulated a general policy against us borrowing their copyrighted works, so if we find a work from WIMA here it's likely to get deleted, unless we know the contributor and the typesetter are one and the same. As most IMSLP contributors use pseudonyms or are shy of stating their full names (I, for example, am one of the rare exceptions) this is more difficult than you would think.

(Some of these issues are covered at: http://imslp.org/wiki/IMSLP:Copyright_Reviewers )

If you're not sure what this site's about, perhaps have a browse through the composer lists and then through the works for specific composers. We're trying to comprehensively cover Western art music, especially the public domain. Manuscripts, published works, facsimiles - whatever. Nothing in MusicXML falls into that category, although having source files like that would be undeniably useful.

Here's an example. There were something like 30,000 symphonies written in Europe in the 18th Century. Most of the ones that have stayed in the concert repertoire are an extremely small proportion of that number. Most were copied by hand and have never been typeset; there have been publications of collections where representative minor works have been typeset, e.g. a volume of symphonies from minor composers in Mannheim, in Salzburg, in Wien, ...

Without some really major developments in automated music OCR for handwritten MSs, MusicXML will only ever scrape the tiniest proportion of the vast amount of stuff that's out there. I don't see either a pro- or anti- stance with respect to MusicXML changing the value of this website. The vast majority of visitors here seem happy with the PDFs.

And yes, most decent music typesetting programs are adding MusicXML import/export to their feature sets.

Regards, Philip