#248714 - 14/03/2005 16:44
Re: empeg file structures
[Re: cushman]
|
member
Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
|
Thanks for the info about displaying a graphic, even though it's not the answer I wanted, still at least I know.
I have overcome the display hurdles - almost.
The hijack popup command requires a time value or it's so quick you can't see it. The test example I copied (by Mark so I assumed it to be correct:-) did not include this. Once I realised this I can display messages.
The rsync progress display requires the different version of libresolv that can be obtained along with the hacked rsync. Once that has been installed the progress is displayed. Since this library already existed and the notes with the replacement version indicate it should be installed 'unless it's already there' I had thought I didn't need it.
A problem I have now found is that if a program (i.e. shell script) is started from the hijack menu (using ;@menuexec, the popup messages don't work - it appears menuexec takes exclusive control of the display which is a shame. Is there a getaround for this?
|
|
Top
|
|
|
|
|
#248715 - 14/03/2005 16:53
Re: empeg file structures
[Re: ukengb]
|
pooh-bah
Registered: 31/08/1999
Posts: 1649
Loc: San Carlos, CA
|
Quote:
However, I am a bit confused. Does Hijack now automatically contain the ext3 patches, or do you have to use the 'special' hijack, an older version which has had the relevant patches applied?
The hijack source includes the patches, but Mark's builds don't have them turned on so you either need to grab the kernel binary from my site or compile your own. Either way you'll want the newer versions of e2fsck and tune2fs from my site.
-Mike
|
|
Top
|
|
|
|
|
#248716 - 15/03/2005 09:16
Re: empeg file structures
[Re: mcomb]
|
member
Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
|
Quote:
Quote:
However, I am a bit confused. Does Hijack now automatically contain the ext3 patches, or do you have to use the 'special' hijack, an older version which has had the relevant patches applied?
The hijack source includes the patches, but Mark's builds don't have them turned on so you either need to grab the kernel binary from my site or compile your own. Either way you'll want the newer versions of e2fsck and tune2fs from my site.
-Mike
I've grabbed e2fsck and tune2fs, but the trouble with using the hijack from your site is that it's not as up-to-date as the current build on Mark's site. I don't think it's practical to compile my own since I would guess that my regular RedHat linux tools wouldn't be suitable.
Any chance that Mark can simply turn on the ext3 support by default? Is thre any reason not to do this?
|
|
Top
|
|
|
|
|
#248717 - 15/03/2005 09:43
Re: empeg file structures
[Re: mcomb]
|
member
Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
|
Quote:
...you'll want the newer versions of e2fsck and tune2fs from my site.
-Mike
Forgot to ask this. Are the above utilities 'backwards compatible'? IOW can they be installed and used for the normal ext2 filesystems should ext3 not be used?
|
|
Top
|
|
|
|
|
#248718 - 15/03/2005 13:06
Re: empeg file structures
[Re: ukengb]
|
carpal tunnel
Registered: 29/08/2000
Posts: 14493
Loc: Canada
|
Quote:
Any chance that Mark can simply turn on the ext3 support by default? Is thre any reason not to do this?
Yes, a very good reason: it takes up significanly more kernel memory. Kernel memory is non-pageable, so it becomes totally unavailable to userspace. Which means that the memory-strapped player software will have even less to work with.
Not an issue on my 32MB players, and not a really bad issue on my v2.0x equipped Mk2a players, but for anyone running v3 software and/or a Mk2 player (12MB instead of 16MB), that memory is NEEDED for other stuff.
Cheers
|
|
Top
|
|
|
|
|
#248719 - 16/03/2005 23:49
Re: empeg file structures
[Re: mlord]
|
pooh-bah
Registered: 31/08/1999
Posts: 1649
Loc: San Carlos, CA
|
Quote:
Yes, a very good reason
Yeah, what Mark said. If you don't know that you want to run ext3 and the consequences of using it you probably shouldn't have it forced on you. That said, I never had a problem running ext3 with the v3 betas on a stock mk2 with something like 10,000 mp3s so I'm not sure just how much extra memory it really uses.
Quote:
Are the above utilities 'backwards compatible'? IOW can they be installed and used for the normal ext2 filesystems should ext3 not be used?
Yep, they are just slightly more current versions of what is already on the empeg. They work fine with ext2 partitions.
Quote:
the trouble with using the hijack from your site is that it's not as up-to-date as the current build on Mark's site
I don't think anything significant has changed in the last few hijack builds which is why I never bothered updating, but it is probably time. Check my site in a half hour or so, I'll throw up a new version.
EDIT: Updated version (hijack 417) now available.
-Mike
Edited by mcomb (17/03/2005 00:16)
|
|
Top
|
|
|
|
|
#248720 - 24/03/2005 13:56
Re: empeg file structures
[Re: ukengb]
|
member
Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
|
Continuing in this thread as it's mainly to do with my rsyncing project.
For testing I have a small set of files locally that I can use to rsync with the empeg - about 20 mp3s and playlists - but the playlists and database files are simply the empty ones as originally created by jEmplode when there was no music at all. IOW, they do not correspond to the files that get uploaded.
After the sync the player app will not start, it just gives a lot of error messages of which this is the end:-
no room for private writable mapping
error: -12
no room for private writable mapping
error: -12
player(134): memory violation at pc=0x0207b864, lr=0x0203a954 (bad address=0x021
78014, code 2)
pc : [<0207b864>] lr : [<0203a954>]
sp : bfffba3c ip : bfffba60 fp : bfffba5c
r10: 2dc6cba0 r9 : 00002dc7 r8 : 00000000
r7 : 00008140 r6 : 0216fe38 r5 : 0216fe38 r4 : 00000178
r3 : 00001028 r2 : 02177fd4 r1 : 00000178 r0 : 0216fe38
Flags: Nzcv IRQs on FIQs on Mode USER_32 Segment user
Control: C0B8117D Table: C0B8117D DAC: 00000015
Function entered at [<0207b83c>] from [<0203a954>]
r6 = 00000178 r5 = 0216FE38 r4 = 02177F7C
Function entered at [<0203a868>] from [<0203aebc>]
r10 = 02175818 r9 = 02112374 r8 = 0214D808 r7 = BFFFFD88
r6 = 0214EF4C r5 = 02175808 r4 = BFFFFD80
Function entered at [<0203aea8>] from [<02061a54>]
The first line gets repeated a LOT first.
I thought that the player app would rebuild the appropriate files if there was a mismatch, why is it having such a problem here?
My intention is that the correct and matched playlist and database files will be uploaded at the same time with the music files and playlists, so this shouldn't be a problem in any case, but I want to get some idea of what's going on here.
Any suggestions as to what is the problem for the player?
|
|
Top
|
|
|
|
|
#248721 - 24/03/2005 14:21
Re: empeg file structures
[Re: ukengb]
|
carpal tunnel
Registered: 18/01/2000
Posts: 5683
Loc: London, UK
|
Quote:
no room for private writable mapping
error: -12
You're out of memory. -12 is ENOMEM.
_________________________
-- roger
|
|
Top
|
|
|
|
|
#248722 - 24/03/2005 14:53
Re: empeg file structures
[Re: Roger]
|
member
Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
|
Quote:
Quote:
no room for private writable mapping
error: -12
You're out of memory. -12 is ENOMEM.
I sort of guessed that, but why would it have such a problem with 6 playlists and 4 mp3 files (and their tags files too)?
Could it be down to the mismatch between the files and what is listed in the 'database' and 'playlists' files, or must it be a problem with the files themselves? As far as I can tell the files are structured correctly, but have I got en error in there somewhere?
What is likely to cause the player app to run out of memory?
|
|
Top
|
|
|
|
|
#248723 - 24/03/2005 15:57
Re: empeg file structures
[Re: Roger]
|
member
Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
|
In fact, it is just the subdirs in the fids folders that are causing the problem.
empeg:/empeg# ll fids?/
fids0/:
total 2
drwxrwsr-x 2 root 60 1024 Mar 7 01:07 _00000
drwxrwsr-x 2 root 60 1024 Mar 7 01:12 _0a2c2
fids1/:
total 2
drwxrwsr-x 2 root 60 1024 Mar 7 01:08 _00000
drwxrwsr-x 2 root 60 1024 Mar 7 01:12 _0a1c2
I am putting all the files into such subdirs just like fidsift does (at least I hope I'm doing it right) and with just these, i.e. with or without any files in them the player has this memory problem.
Is there something else that needs to be done to let the player know it should be using these 'fidsifted' directories?
|
|
Top
|
|
|
|
|
#248724 - 24/03/2005 16:09
Re: empeg file structures
[Re: ukengb]
|
carpal tunnel
Registered: 13/07/2000
Posts: 4180
Loc: Cambridge, England
|
Quote:
drwxrwsr-x 2 root 60 1024 Mar 7 01:12 _0a2c2
Woah, how big are your FID numbers? The car-player expects the FID numbers to be smallish; the database takes up one byte per unused FID below the maximum, and so the maximum wants to be pretty small. Optimum memory usage occurs when the FIDs are dense from 0x100 onwards.
Peter
|
|
Top
|
|
|
|
|
#248725 - 24/03/2005 22:11
Re: empeg file structures
[Re: ukengb]
|
carpal tunnel
Registered: 29/08/2000
Posts: 14493
Loc: Canada
|
Those don't look like the expected naming structure for subdirs, either.
-ml
|
|
Top
|
|
|
|
|
#248726 - 25/03/2005 08:09
Re: empeg file structures
[Re: mlord]
|
member
Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
|
Quote:
Those don't look like the expected naming structure for subdirs, either.
-ml
Well, going by what you've said in the past, the fid (in hex) is padded to 8 digits with zeros to the left and the folder is then _xxxxx (first 5 digits) with the remaining 3 digits used for the filename.
Is that not correct?
|
|
Top
|
|
|
|
|
#248727 - 25/03/2005 08:22
Re: empeg file structures
[Re: ukengb]
|
carpal tunnel
Registered: 18/01/2000
Posts: 5683
Loc: London, UK
|
Quote:
Is that not correct?
Quote:
_0a2c2
Well, that's fine, but you appear to have a FID numbered a2c2??0.
Assuming that this is the case, you've got something like 10 million unused FIDs. This is, as Peter says, 10Mb of RAM just to hold a single FF byte for each. There's no way that that's not gonna be a tight fit.
_________________________
-- roger
|
|
Top
|
|
|
|
|
#248728 - 25/03/2005 08:39
Re: empeg file structures
[Re: peter]
|
member
Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
|
Quote:
Quote:
drwxrwsr-x 2 root 60 1024 Mar 7 01:12 _0a2c2
Woah, how big are your FID numbers? The car-player expects the FID numbers to be smallish; the database takes up one byte per unused FID below the maximum, and so the maximum wants to be pretty small. Optimum memory usage occurs when the FIDs are dense from 0x100 onwards.
Ah, I may have missed an important factor here.
Are you saying that the fids really need to be contiguous (from 0x100) as the database file has to include the space for EVERY FID, whether it exists or not?
In that case I'm way off track. I start from 0x100 for the Master Playlist and then skip to 0x200, 300 etc for other top level playlists (0x640 max), then from 0x1000 for Art/Alb playlists (lots of them), then from nearly 2 million for the actual music files since the numbers are based on their inode, plus a constant (1 m.) to ensure there's no clash with the other numbers.
This means that there is no clash of numbers, but obviously there are big gaps in the range of fids, so if the database stores ALL the fids, then I'm being very wasteful.
A similar scheme works perfectly for my Rio Receiver music server, but that doesn't seem to care about discontiguous fid ranges, they're just ID numbers.
As you may remember, all my data is taken from iTunes which of course has its own numbering scheme for playlists and music data files, but irritatingly it has a tendency to re-number them, particularly after an iTunes update, so they're not much use for this purpose.
Are the problems with discontiguous fid ranges limited to the database?
|
|
Top
|
|
|
|
|
#248729 - 25/03/2005 09:45
Re: empeg file structures
[Re: ukengb]
|
carpal tunnel
Registered: 13/07/2000
Posts: 4180
Loc: Cambridge, England
|
Quote:
Are you saying that the fids really need to be contiguous (from 0x100) as the database file has to include the space for EVERY FID, whether it exists or not?
Yes. They don't have to be completely contiguous, but they need to be a lot less sparse than yours. The space taken up by a completely unused FID is only one byte, but if you've got several million unused FIDs that means you're still in trouble.
Quote:
Are the problems with discontiguous fid ranges limited to the database?
Yes. For instance, the Rio Karma, which has a rather smarter database, doesn't suffer from this problem.
Peter
|
|
Top
|
|
|
|
|
#248730 - 25/03/2005 10:07
Re: empeg file structures
[Re: peter]
|
member
Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
|
Oh well, we live and learn. I hadn't appreciated this was the situation since the Rio Receiver is so different.
I'll just have to work on a new numbering scheme.
Thanks for the info.
|
|
Top
|
|
|
|
|
#248731 - 29/03/2005 14:28
Re: empeg file structures
[Re: peter]
|
member
Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
|
If there's no file(s) for a particular fid, does there have to be an entry in the database file? If so, what needs to be entered?
BTW, what's up with Roger's site? I want to re-acquaint myself with his explanation of the file structures, but it seems to be down.
|
|
Top
|
|
|
|
|
#248732 - 29/03/2005 16:45
Re: empeg file structures
[Re: ukengb]
|
carpal tunnel
Registered: 13/07/2000
Posts: 4180
Loc: Cambridge, England
|
Quote:
If there's no file(s) for a particular fid, does there have to be an entry in the database file? If so, what needs to be entered?
Entries must be present in the database for every consecutive fid from the smallest (which should be 0x100) to the largest. The database entry for a fid that has no files is just the terminating 0xFF byte.
Peter
|
|
Top
|
|
|
|
|
#248733 - 30/03/2005 06:53
Re: empeg file structures
[Re: peter]
|
member
Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
|
Quote:
Quote:
If there's no file(s) for a particular fid, does there have to be an entry in the database file? If so, what needs to be entered?
Entries must be present in the database for every consecutive fid from the smallest (which should be 0x100) to the largest. The database entry for a fid that has no files is just the terminating 0xFF byte.
Thanks. Figured it would be, but having missed a fairly major aspect of this, I thought I'd double check:-)
BTW, I still can't connect to Roger's site. Is it down? Anyone any news on that?
|
|
Top
|
|
|
|
|
#248734 - 30/03/2005 19:45
Re: empeg file structures
[Re: ukengb]
|
carpal tunnel
Registered: 24/12/2001
Posts: 5528
|
It's hosted on his ADSL line at home.
|
|
Top
|
|
|
|
|
#248735 - 02/04/2005 12:17
Re: empeg file structures
[Re: tman]
|
carpal tunnel
Registered: 18/01/2000
Posts: 5683
Loc: London, UK
|
Quote:
It's hosted on his ADSL line at home.
And it appears to have fallen over while I was away in the French Alps with no way to check on it. It's up now.
_________________________
-- roger
|
|
Top
|
|
|
|
|
#248736 - 02/04/2005 12:20
Re: empeg file structures
[Re: peter]
|
carpal tunnel
Registered: 18/01/2000
Posts: 5683
Loc: London, UK
|
Quote:
Entries must be present in the database for every consecutive fid from the smallest (which should be 0x100) to the largest.
Entries must be present in the database for every consecutive FID from 0x0 to the largest. The first entry (0x0) in the file has a type of "illegal", and the next 15 (0x10 to 0xF0) are a single 0xFF byte.
_________________________
-- roger
|
|
Top
|
|
|
|
#248737 - 02/04/2005 14:22
Re: empeg file structures
[Re: Roger]
|
carpal tunnel
Registered: 13/07/2000
Posts: 4180
Loc: Cambridge, England
|
Quote:
Entries must be present in the database for every consecutive FID from 0x0 to the largest. The first entry (0x0) in the file has a type of "illegal", and the next 15 (0x10 to 0xF0) are a single 0xFF byte.
 Quite right. That's what happens with not rereading the emptool sources before posting.
Peter
|
|
Top
|
|
|
|
|
#248738 - 02/04/2005 16:05
Re: empeg file structures
[Re: Roger]
|
carpal tunnel
Registered: 20/12/1999
Posts: 31597
Loc: Seattle, WA
|
Quote:
And it appears to have fallen over while I was away in the French Alps
How was the snow? 
|
|
Top
|
|
|
|
|
#248739 - 02/04/2005 16:47
Re: empeg file structures
[Re: tfabris]
|
carpal tunnel
Registered: 18/01/2000
Posts: 5683
Loc: London, UK
|
Quote:
How was the snow?
Pretty poor, to be honest. We had plenty of sun and some fresh snow on the last day, though.
_________________________
-- roger
|
|
Top
|
|
|
|
|
#248740 - 02/04/2005 16:58
Re: empeg file structures
[Re: Roger]
|
carpal tunnel
Registered: 24/12/2001
Posts: 5528
|
Quote:
Quote:
How was the snow?
Pretty poor, to be honest. We had plenty of sun and some fresh snow on the last day, though.
Meh. Still better than my skiing holiday in January where I injured myself on the first hour of the first day. I didn't even fall over! 
|
|
Top
|
|
|
|
|
#248741 - 20/04/2005 09:44
Re: empeg file structures
[Re: ukengb]
|
member
Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
|
I have re-written my uploader to utilise fids from 16 upwards with no gaps in the range, except due to deletions and these get re-used. So that all looks good now.
But I've still got a major issue.
If the music server is the rsync client so the empeg is the server and the player app is still running, then after the sync is finished I CANNOT set it ro. If I log in via telnet and try to set it ro it just reports "drive0 is busy". If I quit the player at the serial terminal and then exit so the player starts again, all is now OK.
I want to control the whole thing from the music server, i.e. have no interaction on the actual empeg and if I simply re-start the player app, it doesn't clear the 'drive0 is busy' error.
So, what might be causing drive0 to be busy and how can I circumvent that?
Should I sync while the player app is NOT running. This seems to work, but how can I stop it for the duration of the sync? Connecting via serial and quitting the app to start a shell session is not an option. As I said, I'm trying to get this to work without having to interact with the empeg.
Any ideas?
|
|
Top
|
|
|
|
|
#248742 - 20/04/2005 10:41
Re: empeg file structures
[Re: ukengb]
|
addict
Registered: 14/11/2000
Posts: 474
Loc: The Hague, the Netherlands
|
if you have "killall" installed on your player, you could remotely invoke killall -INT player to stop the player, and killall -HUP bash to restart it.
Pim
|
|
Top
|
|
|
|
|
#248743 - 21/04/2005 06:48
Re: empeg file structures
[Re: pim]
|
member
Registered: 30/04/2003
Posts: 136
Loc: United Kingdom
|
Quote:
if you have "killall" installed on your player, you could remotely invoke killall -INT player to stop the player, and killall -HUP bash to restart it.
Have now:-)
However, last time I checked I found a thread where everyone was discussing how to stop the player, but the general consensus of opinion was that you couldn't as it would immediately re-spawn.
I'm pleased to have now got the solution. Thanks.
|
|
Top
|
|
|
|
|
|
Previous Topic
Index