Melodeon.net Forums

Please login or register.

Login with username, password and session length
Advanced search  

News:

Welcome to the new melodeon.net forum

Pages: [1]   Go Down

Author Topic: EasyABC eliminating blank lines  (Read 2207 times)

0 Members and 1 Guest are viewing this topic.

Stiamh

  • Old grey C#/D pest
  • Hero Member
  • *****
  • Offline Offline
  • Posts: 3538
    • Packie Manus Byrne
EasyABC eliminating blank lines
« on: May 22, 2018, 06:51:50 PM »

Relatively recent convert to EasyABC here after years of hand-coding.

Puzzling behaviour in one particular file containing 400+ tunes. Every time I open this particular file in EasyABC, in the code window the ABC code is displayed with no blank lines between tunes (before the X: field). The spaces are there, and if I close the file without making changes, and open it up in my text editor, they are still there.

But as soon as I make one change to file and save it, all blank lines are gone. It's getting a bit tiresome reinserting all these hard returns with a search/replace command.

The strange thing is that it doesn't do this with any other files that I work on with it. I have just opened another file that has about 1600 tunes in it and this doesn't happen. So presumably the size of the file is not the issue. Anybody else had this trouble?

Investigating this I realized that easyABC only numbers lines up to 9999 and then starts again. Pretty lame! (The problem file has over 12,000 lines in it).

Gena Crisman

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 1041
  • 🇬🇧
Re: EasyABC eliminating blank lines
« Reply #1 on: May 23, 2018, 02:33:33 AM »

So as best as I can tell (at 2 am), given that EasyABC is open source it should be possible to work out what is going wrong here.

Looking in easy_abc.py, it looks like that on line 3048 (in the load method) it runs the following code:
Code: [Select]
text = re.sub('\r+', '\r', text)

This code is attempting to resolve a line ending conflict between unix based and windows based operating systems. This line of code would attempt to replace any chained '/r' characters. After that, some other code checks to see if any /ns exist in the file, and if there are none, converts all '\r' into '\n'. It selects different code if the flag '__WXMAC__' is set, which it gets from its user interface library, wxWidgets. This is... probably not the right way of detecting which line endings to use, and is probably not the right way of fixing the line endings, either.

So, if your file contains blank lines, was saved on a unix style line ending using platform, and you are either not using or a mac or __WXMAC__ is not set for some reason or another, the character stream would be '\r\r\r\r' etc for all your blank lines: this code would replace this with a single \r, removing the blank lines from it - this is why adding a space prevents this from occurring.

May I ask what operating system you're using? And, is it possible to share the file? What is your 'text editor' - if it's smart, it might not be showing + might be preserving line endings for platforms that don't match your own, which might be why you are experiencing this with only one file.
Logged

Stiamh

  • Old grey C#/D pest
  • Hero Member
  • *****
  • Offline Offline
  • Posts: 3538
    • Packie Manus Byrne
Re: EasyABC eliminating blank lines
« Reply #2 on: May 23, 2018, 03:15:40 AM »

Hi Gena,

Very interesting! My OS is Windows 7 and my text editor is EditPad Pro. The problem abc file is part of a collaboration project with multiple contributors, but all the various bits of it have been worked on by one particular contributor besides myself.

The files that don't exhibit this line-dropping behaviour are all ones I have compiled myself. So maybe the other fella is on UNIX - I'll find out.

I'm not familiar with \r and \n characters... what language is that? (:)

I could reinsert blank lines with a space in them but I'd rather find a more elegant solution. What do I need to strip out of the file to purge it of all traces of UNIX? Replace every single return character?

I can't share the problem file publicly just yet but I'd be happy to send it to you for sleuthing purposes. Send me a pm with your email address if you'd care to look at it.

Edited to add: I think I see the problem. Nearly all the line-end characters in the problem file are hex 0D (CR) only. Whereas in the ones that don't exhibit the line-dropping behaviour, they are 0D 0A (CR LF).

I have also just realized that EditPad Pro is making decisions about what type of file it is dealing with. When I hit Enter in the problem file, it inserts 0D. But in a non-problem file, 0D 0A! It's not quite 2am here but time to stop.  :|glug

Edit 2: Copy contents of abc file from EditPad into Word. Copy it back into a blank EditPad file. All line ends are now 0D 0A. Save as ABC and open in EasyABC. Problem does not occur.  :|glug :|glug :|glug Thank you!
« Last Edit: May 23, 2018, 03:46:15 AM by Stiamh »
Logged

Gena Crisman

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 1041
  • 🇬🇧
Re: EasyABC eliminating blank lines
« Reply #3 on: May 23, 2018, 11:57:26 AM »

Glad you have it sorted, I imagine based on the edits it's been a bit of a journey in learning about line endings, sorry that I didn't supply more links to things but I want sure exactly your situation.

Often, fancier text editors will auto detect line endings and stick with them, as if eg you're editing a Unix script on another machine using a Windows system you'd want it to still work right afterwards. There may be an option to change the line endings one way or the other in edit pad Pro, but it's one of those things you don't know you need until you do.

The solution I used to use was use wordpad on Windows and resave the file with that, as pretty much every Windows pc has wordpad, and it will resave to a clean text file without too much persuading, but your solution is also perfectly viable, too.

Do watch for the problem creeping back in if the file is worked on by eg a mac user, as its not impossible that the reverse conversion may take place, resulting in the lines being wiped again when you reopen it in easy abc.

I would have a go at fixing it, since this clearly isn't behaving itself, and submitting that change to the author but I'm not sure if they're expecting that kind of thing, and sometimes building projects locally can be a bit of a quest. It might be something that has to be lived with for a while.
Logged

Stiamh

  • Old grey C#/D pest
  • Hero Member
  • *****
  • Offline Offline
  • Posts: 3538
    • Packie Manus Byrne
Re: EasyABC eliminating blank lines
« Reply #4 on: May 23, 2018, 12:25:43 PM »

Many thanks for poking around under the bonnet and identifying the problem behaviour in EasyABC. No need to apologise for not including links - you set me off in the right direction and learning something new is always good - about other operating systems, and indeed about the trusty text editor I've been using for decades. I couldn't have imagined that it would make the distinction between CR and CR/LF characters, even in search-and-replace operations, based on the file contents.

As for suggesting a fix, there is a sourceforge.net discussion forum moderated by JW de Jong, who seems to be supporting the package since Nils Liberg stopped doing so. I could relate my experience but the tale might be more convincing coming from you, since you could talk more coherently about the problem bit of code.  (:)

PS You are right about the options in EditPad Pro. I have enabled the status bar indicator that tells me that the dominant line-end style in the problem file is not UNIX (LF), but Mac (CR)!! From the help file:

Quote
Convert|To Windows (CR LF)

Converts the active file to the line break style used by Windows and DOS text files. These operating systems terminate each line with a Carriage Return and Line Feed pair. Only the line termination characters are converted. Anything else remains untouched.

To EditPad, it does not matter whether a file uses Windows, UNIX, or Mac line breaks. EditPad even handles files that use a mixture of all three. However, other applications often only support the "proper" line breaks for their environment. Windows applications like Notepad display UNIX text files with everything on one line.

Use the Convert|To Windows command to make sure your file uses consistent Windows line breaks if you'll be using the file with other Windows applications. If the Convert|To Windows menu item is grayed out, that means the file already uses Windows line breaks consistently.

If line break styles matter for your work, use the Status Bar Preferences to enable a status bar indicator to show the file's (dominant) line break style. In EditPad Pro, in the Editor Preferences you can configure Options|Visualize Line Breaks to display the actual line break style for each line rather than a generic paragraph marker.
See also

Convert Menu
Convert|To UNIX (LF only)
Convert|To Macintosh (CR only)
« Last Edit: May 23, 2018, 12:47:14 PM by Stiamh »
Logged

Gena Crisman

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 1041
  • 🇬🇧
Re: EasyABC eliminating blank lines
« Reply #5 on: May 23, 2018, 08:03:59 PM »

So, I was a bit thrown for a loop a bit by your comment, but you are quite right. I swap between OS X and Windows fairly frequently and was... surprised, let's say, to learn there was yet another ridiculous implementation for line endings. While modern Macs use OS X, which is (according to fairly dry stuff) at least unix based, and uses \n for line endings, it turns out that it's OS 9 and earlier using Macs from around the turn of the millennium that use a single \r character for their line endings - a fact I was out of touch with, it seems, but should have noticed the unusualness of given my familiarity with the subject. This particular code, then, may be... really quite out of date - in fact EasyABC's code is littered with conversions to other line ending types for sending to the ABC interpreting binaries it uses for the majority of its functionality.

Anyway, thank you for the link, with your help I have tracked down the current maintainers and have taken a look at their more up to date version/changes, but sadly this code still persists (at a different line number). I'll see if I can draw it to their attention. The project doesn't really seem to be being actively developed, though.

Good luck with the project that you're working on!
Logged

Stiamh

  • Old grey C#/D pest
  • Hero Member
  • *****
  • Offline Offline
  • Posts: 3538
    • Packie Manus Byrne
Re: EasyABC eliminating blank lines
« Reply #6 on: May 23, 2018, 10:47:08 PM »

Thanks again Gena. This discussion has been most useful, and with luck it may benefit other EasyABC users (please shout if it does, any of you).

News of the collaborative venture should be forthcoming before too long. It is the project to transcribe into ABC all the tunes on http://mustrad.udenap.org/lerepertoire.html, with the blessing of the owner of the site. Project initiated by Pete Dunk of this parish, backed up by a team of tireless transcribers. Transcriptions are done, basically. Some final vetting has to be performed and then some work will have to be done on the site to make the new transcriptions and ABCs available.

vof

  • Regular debater
  • ***
  • Offline Offline
  • Posts: 211
Re: EasyABC eliminating blank lines
« Reply #7 on: May 24, 2018, 12:54:44 AM »

Sorry for coming late to this...
So maybe the other fella is on UNIX
...unlikely, but much more likely Linux or Mac OS X (or even, as Gena suggests, OS 9  :o) all of which have quite a lot of shared genes with Unix.

Vince
Logged
Vince O'Farrell
Dino BPII; Hohners: Pokerwork D/G (fettled by Theo; 4th button start, Viennese tuning, and thirds stop by Mike Rowbotham); red Erica G/C; couple of 1040s in C and G project boxes.
Pages: [1]   Go Up
 


Melodeon.net - (c) Theo Gibb; Clive Williams 2010. The access and use of this website and forum featuring these terms and conditions constitutes your acceptance of these terms and conditions.
SimplePortal 2.3.5 © 2008-2012, SimplePortal