allowing lilypond and MusicXML source upload

[Johann Casper Ferdinand F]

It is possible to attach the lilypond source file to the PDF it creates, using pdftk and similar tools. Then, the LY file will not get lost, and will follow the PDF wherever it goes.

Notation source files do offer certain advantages over scanned scores for particular uses. They allow for correction of errors. They allow for the addition of fingerings and other editorial content. They allow the page size and layout to be changed. They allow parts to be extracted from scores and simple arrangements to be made by separating voices.

True, IMSLP is mainly about scanned files, but typesets are important too. See for example the Gyrowetz typeset I recently uploaded. No modern edition besides my own is available, and I based my edition on the 1700s-era published edition in the Sibley library. The 1700s-era edition is parts only (no score available), is poorly edited, uses an oversize paper size that won't fit in standard printers or notebooks, and uses notational conventions that are difficult to sightread for modern performers. My typeset addresses all that, and I think it's more valuable than the original score, as a practical performing edition.

The Mutopia project, which collects and distributes lilypond source files, has various barriers to participation. Their copyright clearance requirements are strict and are difficult for some projects to meet. They impose a life+70 requirement on the editor of the source edition upon which the typeset is based even if editorial contents are not preserved, and even if the source edition was only published in the U.S., which rules out a substantial number of scores. They also require MIDI files, which many typesetters have no inclination to provide, and they require all typesets to produce both letter and A4 output. All things that make sense given their mission but they are not the ideal destination for every typesetter or for every typeset.

[Carolus]

That's an interesting idea. Having some sort of source file attached to a PDF could be very useful. Is there any chance of such a file not being readable (as a normal PDF) by a user who only had the Adobe reader installed?

[Johann Casper Ferdinand F]

No. It's part of the standard, and Reader and other viewers ignore it. I believe that Acrobat Professional allows attachments to be viewed, added, and removed.

You can take a look at any of the typesets I've posted if you're curious. They all have lilypond files attached, as noted in the comments.

[Johann Casper Ferdinand F]

Everything has been said, but:

Typesets are not something that IMSLP wants to promote. Most unprofessional typesets (which is what these are) are far from anything resembling publishable standards - they are ugly.

Reading through the thread, I would like to address this misconception.

Yagan is correct that there are a large number of typesets that are not pleasing to the eye or that are fraught with errors. This is especially true of older typesets made with MusicTeX and its variants, which even in skilled hands do not produce scores of high visual quality. Recent software, in capable hands, produces results that rival the best published editions, and there are careful typesetters whose accuracy is as good as any commercial publisher. High quality output was one of the design goals for Lilypond, and while there are exceptions, scores from typesetters using Lilypond are dense and highly readable.

I have the fruits of two major typesetting projects on IMSLP, and invite Yagan and any others who believe that typesets are of uniformly poor quality to review them. I am not alone, as there are numerous contributors to the Mutopia Project whose work is comparable to mine. A few contributors to WIMA, notably Pierre Goudin, produce work to an equally high standard using commercial packages instead of Lilypond.

[daphnis]

Re. attachments - Acrobat Pro can indeed view and make attachments. Adobe reader can view these attachments, although I'm not certain about 3rd party viewers.

Re. typesets - Without plunging into the pros and cons of each typesetting package, although I will say Lilypond produces good results for less-complex scores, but I don't believe composing and laying-out by command is an intuitive enough interface to handle the complexities of later works such as Debussy et al, I would either vote for those typeset "sources" to be attached to the PDF or let the author be contacted if the typeset original is wanted.

[Lyle Neff]

May I suggest that, if some PDFs are going to include source-file attachments, there should be a alternative "plain" PDF without the attachment available as well to download in order to save bandwidth.

(Same thing with including color title pages, as I think I've mentioned elsewhere.)

[daphnis]

Lyle, there really shouldn't be any noticeable difference so long as only typeset "source" files are included. They are generally very small by comparison, so I don't think there's any bandwidth concern here.

[Johann Casper Ferdinand F]

Size of the attachment, generally, is not an issue.

Bear in mind that PDFs produced by typesetting programs are typically smaller than PDFs that result from scans, and the source files for typesetting programs are typically small compared to the size of the PDFs.

A recent example I happen to have at hand is 62 pages and runs 1.21MB without the attachments, and 1.23 MB with.

For comparison, higher quality scans of that length typically run 3-6 MB.

[Yagan Kiely]

Re. typesets - Without plunging into the pros and cons of each typesetting package, although I will say Lilypond produces good results for less-complex scores, but I don't believe composing and laying-out by command is an intuitive enough interface to handle the complexities of later works such as Debussy et al, I would either vote for those typeset "sources" to be attached to the PDF or let the author be contacted if the typeset original is wanted.

Exactly my thoughts.

Although I don't want to restrict typesets on IMSLP, I don't think it is beneficial to IMSLP to promote it. Regardless of whether there are many extremely competent typesetters out there (which I do not deny), there are even more under-competent entrepreneurs of typesetting. What is stopping them from uploading there horrible scans? Also you have to remember, the overwhelming majority of people who use IMSLP, will not be people overly competent in Lilypond. Universities and schools are equipped in Sibelius or Finale, and most people would also prefer them (GUI), a lot of Lilypond users are not the generic music student, and don't think un-simplifying IMSLP by including any notation software file is justifiable by the minority that would use it. If it can be easily added to a PDF as an attachment without any complex strings attached I'd be completely fine with that. Alternatively, people could contact the typesetter, I doubt that any typesetter will be overwhelmed with requests.

[pml]

This thread is becoming more and more interesting as it goes on!

I haven't actually read the ISO standard defining the Portable Document Format, but JCFF's suggestion above regarding attachments is extremely interesting, and one I will be investigating for my own projects.

As a sometime perpetrator of under-competent typesetting, I've been faced with a quandary which I will describe as the Microsoft dilemma: I know this stuff I've been working on for years still has lots and lots of bugs in it, and might not work properly when someone comes to use it, so should I hold onto it privately until I've fixed all of the numerous problems, or should I just dump it out into the world and let everyone else deal with it?

I won't be drawn on making invidious comments about other people's typesets, so I'll confine myself to my own projects. In the main, there isn't a lot of money going around to pay typesetters to do either competent type-setting, or competent proof-reading. As a result some of my own typesets have been largely to please myself, and if anyone else happens to find them useful, that's a bonus.

The Mozart Great Mass in C is perhaps the most notorious of my editions in this respect. Back in 2001 there was a request from someone on the (very lengthy) CPDL request list who wanted the vocal score of the Kyrie from that great work. So having a long interest in the piece anyway, and having the Dover volume of six of the Mass ordinaries, I went about typesetting just the VS of that movement in order to fill the request. But then... that short Gloria movement that follows immediately afterward (and quotes Handel's Hallelujah chorus) was equally deserving of being given a typeset... so I gave in... and the next year I performed the work with the Melbourne Philharmonic, so it seemed like a good time to do MIDI or Scorch versions of all of the choral movements... and so on...

... and eight years later, the typeset consists of 200+ pages of full score, about 100 pages of vocal score, and over 500 pages of orchestral parts for 24 different instruments – slightly more than Mozart's scribes needed to generate, since they used natural horns and trumpets in a variety of crooks, whereas I have made concessions to the normative brass keys in common use.

In the meantime, while that typeset was gradually being assembled, the Microsoft dilemma I described above could be re-couched in the following terms: even if not all of the work happened to be available, was there any use in making available whatever excerpts of the Mozart were ready to hand at the time – possibly with mistakes, and with a decided lack of proofreaders – with the intention of going back and finishing off the job later? Certainly in terms of the users of CPDL, the answer to that was a thundering "yes".

I think the most embarrassing and incompetent error that went undetected for a strikingly long period of time (from the original release of the score in March 2002 through three subsequent revisions until May 2005!), was in the very first movement to be attempted, where the final notes of the soprano soloist omitted the grace notes before the end of the long cadential trill. However, it is a startling fact that in all that time, during which I received numerous e-mails regarding the typeset, no one observed that those two little notes were missing... Hmm...

On the other hand, my scores for Tallis' 40-part motet Spem in alium are one of the finest things I've done, and at Perlnerd's prompting I will be uploading my scores here shortly, in effect mirroring the set over at Choral Public Domain Library.

I am well aware of the vagaries of traditional typesetting as well as the specific rules that govern musical typesetting, and have at times experimented with my own custom house style: although I am heartily sick of reading scores where the main text font is Times New Roman, many of my scores have gone down that particular path of least resistance, just because the font is so damn ubiquitous when the score is to be distributed and printed on virtually any type of laser printer you can point a stick at, which one would hope will always have that font stored permanently in memory.

It's a little bit embarrassing when you go to the effort of finding a nice serif font like Bembo, or Goudy Old Style, or something a little out of the ordinary, and go to the trouble of managing ligatures, proper quotation marks, em-dashes and the like, only to find that when Jo Bloggs from Idontknowhereistan sent the PDF to his laser printer, the text came out like:

Cruci?xus, by Antonio Lotti (øÞ.1667?1740) instead of
Cruci?xus, by Antonio Lotti (ca.1667–1740), and the noteheads have all turned from the proper music font into something looking like: ? ? ?

Anyway, I've just been called to bed by my long-suffering partner, so I must say adieu for tonight.

Regards, PML

[daphnis]

PML, I don't think the core issue here is the scholarly or otherwise professional quality of typesets but rather the inclusion of them all together. But to address your last post, I think it's better to have any typeset than none at all, errata notwithstanding. Let's not forget because something was set down in stone, and by that we mean zinc engraving plates, does not at all absolve it of errata. I'm reminded of Ricordi's first edition of orchestral parts to Verdi's La Traviata specifically as possibly representing the worst quality and accuracy of any orchestral parts I've encountered to date. The second issue, which I introduced not to seer this topic off the road, was the merit of certain typesetting programs with certain types of literature.

So in any case, to just recap. my position on the issue at hand: I think source typeset files should be attached to the generated PDF or otherwise the typesetters email should be plainly available in the Misc. Notes field of the submitted work. We need to strive to maintain our format consistency, which is why we did away with ZIP archives, DejaVu files and other proprietary formats a while back.

[pml]

Perhaps its not the core issue, but Yagan's point – at least, if I read it correctly, I think was what he was saying – was that he didn't want any especial encouragement of new typesets on IMSLP, and allowing other file formats such as .ly, .xml appears to open that door.

Although I do my work in Sibelius, I haven't uploaded any of my source files (as far as I recall – which sometimes doesn't stretch very far!). In fact, I think I would actually refrain from uploading the actual .sib files, even though that format is allowed, and follow JCFF's method of attachment them to the PDFs, if that is feasible.

For many reasons I am hesitant to upload source files for my work, one of them being that I've seen them re-appear on other websites under dubious circumstances, which involved the "customer" paying someone other than me, who had not made any prior arrangement to recompense me for my labour...

Anyway, I've made my views on the topic well known in numerous other places. (e.g. here...)

Best, PML

[hughsung]

Just throwing my 2 cents in here as both a Lilypond AND an IMSLP forum newbie. First off, I absolutely love the incredible resources here in IMSLP and am very grateful for the open source spirit that makes the great masterworks of music literature freely available to all! I will always deeply appreciate the repository of scanned public domain scores, but every so often the image scan quality can be somewhat difficult to read. A caveat: i stopped using paper scores about 6 years ago and use only computers to read all of my music, along with special foot pedals to turn the digital pages hands free. Since i'm viewing music exclusively on computer screens, i've really come to appreciate how clean the Lilypond typeset tends to be with the scores i've found on the MutopiaProject.org website. I'm slowly learning how to typeset my own scores and hope to work on more as time allows (graphical GUI for realtime Lilypond output, please?) While my old versions of Finale and Sibelius always printed to paper just fine, their PDF exports somehow never looked that good on the computer screen - Lilypond's PDF's look much better to my eyes. Perhaps the newer versions of Finale and Sibelius have improved PDF printers? I'm hoping to save up for the newest versions soon to test that out.
As the world moves towards digital e-Readers like the Kindle (larger version coming out this week, i hear) and even the iPhone, i'm hoping that this will mean a renewed interest in music literacy and greater appreciation for the treasures found here in IMSLP!

[Yagan Kiely]

9.3" (?) Kindle I believe. Also, it's rumoured Apple ordered some screen in-between the iPhone and 13"MacBook, so that could be interesting.

I don't mind having typesets, I just don't want to promote it - at all. The more promoted, the more likely the really bad (or even average) typesets will be uploaded.

[haydenmuhl]

Bump.

I've just discovered Lilypond and would like to add my voice to this discussion. I'm also wondering if any decisions about this have shaken out since the last time this thread was active.

IMO, the source for any Lilypond typesetting should be included with the PDF upload. Not including the source precludes the possibility of other users correcting transcription errors. Including the source presents no problems for those people unfamiliar with Lilypond.

I think one place Lilypond could be very useful is for those pieces where high quality scans are not available. We could have a list of pieces that need transcription, and anyone who wanted to try their hand at typesetting could give it a go. Upload the source, and in true wiki style, allow community members to come along afterward and make tweaks and edits.