Unoffical empeg BBS

Quick Links: Empeg FAQ | RioCar.Org | Hijack | BigDisk Builder | jEmplode | emphatic
Repairs: Repairs

Topic Options
#335062 - 16/07/2010 16:59 App storage revisited
tonyc
carpal tunnel

Registered: 27/06/1999
Posts: 7058
Loc: Pittsburgh, PA
(Mainly a question for Mark, but opening the floor for comments as well.)

As I'm doing this XM radio thing, it occurs to me that it would be really handy to have a facility in Hijack to store and retrieve settings such as preset banks, the last channel that the receiver was tuned to, etc. When we talked about this a while back, the two approaches I remember being discussed were (a) stealing some unused bits from the player save area, and (b) a kernel call to actually write blocks of data somewhere on one of the disk partitions (maybe the hda3 dynamic data partition?) without actually mounting the disk.

I guess I'm wondering what the possibility of implementing either of these approaches in Hijack would be. Obviously, using the flash save area would only give a few nybbles here and there, not enough to do everything I'd like to do with this app, but better than nothing. Is the latter approach something that would take a lot of work? Does the player use the entire dynamic data partition, or is there an area that's unused that we could steal for blocks of app-specific data?
_________________________
- Tony C
my empeg stuff

Top
#335072 - 16/07/2010 20:52 Re: App storage revisited [Re: tonyc]
tman
carpal tunnel

Registered: 24/12/2001
Posts: 5528
The unpatched player won't use more than the default 16MB or whatever it was of the dynamic data partition. If you patched it using Mark's tool then it'll use up all or most of it. You'll have to wait for him to see what the exact calculation is.

Isn't there an unused partition on the HD?

Top
#335073 - 16/07/2010 20:59 Re: App storage revisited [Re: tman]
tonyc
carpal tunnel

Registered: 27/06/1999
Posts: 7058
Loc: Pittsburgh, PA
Depends on what the meaning of "unused" is. /dev/hda2 isn't used on a stock install, but I've always used it as a /usr partition for apps and so forth. I guess I'd be willing to move all my stuff to the music partition if using hda2 ended up making implementation of this easier.
_________________________
- Tony C
my empeg stuff

Top
#335074 - 16/07/2010 21:06 Re: App storage revisited [Re: tonyc]
tman
carpal tunnel

Registered: 24/12/2001
Posts: 5528
Unused as in stock install. I think some people turned it into swap as well along with your usage of it as extra app storage.

Might be a little hairy but if you allocated a dummy FID of some kind then you could use the dynamic data partition record for that as storage. The translation from FID -> offset is set already.

In theory you could integrated it into the player by doing this as well. Add extra dummy FIDs that correspond to your stations. If you see the player playing one of your fake FIDs then you'll take control. You'd have to work out some way of creating these FIDs though since Emplode obviously won't.

I don't know if the player will like you doing this though behind its back and whether a sync will blow away your data.

Top
#335080 - 16/07/2010 23:01 Re: App storage revisited [Re: tman]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
I think there's a fair bit of space still left in the flash savearea -- and since the player software is "done", anything left now is fair game

Top
#335081 - 16/07/2010 23:04 Re: App storage revisited [Re: mlord]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
To use the hard disk, one really good approach might be to just create a file manually (or have the app remount,rw create the file and them remount,ro again).

Then use the FIBMAP ioctl() call to find out where exactly the file is stored -- LBA numbers. With that info, the app can open the raw disk r/w (not the filesystem), and just read/write those blocks directly.

Takes a bit of fuss to get the correct absolute sector numbers from the FIBMAP data, but hdparm knows how to do it (source available).

I'm traveling right now, and trying not to program (much). smile

Top
#335082 - 16/07/2010 23:25 Re: App storage revisited [Re: mlord]
tonyc
carpal tunnel

Registered: 27/06/1999
Posts: 7058
Loc: Pittsburgh, PA
Well, then, consider these feature requests for your copious free time when you return from travel. smile

I think my XM app could make use of both types of storage, using the save area for small bits like "last channel number tuned to", and using the disk for banks of presets, lists of favorite artists to alert on, etc. Definitely not enough real estate in the save area for those things.
_________________________
- Tony C
my empeg stuff

Top
#335110 - 18/07/2010 20:22 Re: App storage revisited [Re: tonyc]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
The disk portion is purely userspace. smile

Top
#335112 - 18/07/2010 21:34 Re: App storage revisited [Re: mlord]
tonyc
carpal tunnel

Registered: 27/06/1999
Posts: 7058
Loc: Pittsburgh, PA
Originally Posted By: mlord
The disk portion is purely userspace. smile

Oh, right. Well, the stealing bytes from the flash save area thing would certainly be nice to have. I'd want the currently-selected XM station (8 bits) to be saved when power's lost.

For now, I've just got the presets being stored in a flat file that needs to be edited when the disk is mounted. I don't think I'd fare well with trying to implement the raw writes to the disk thing, but I might try it some day once I buy a disk for my development empeg.


Edited by tonyc (18/07/2010 23:04)
_________________________
- Tony C
my empeg stuff

Top
#335121 - 19/07/2010 14:22 Re: App storage revisited [Re: tonyc]
tonyc
carpal tunnel

Registered: 27/06/1999
Posts: 7058
Loc: Pittsburgh, PA
What if the player was patched with the max fid utility to use n-x sectors, where n is sufficiently large for storing the user's running order and bookmarks, and x is a small number of sectors that would be available for saving of application data? Wouldn't the task of writing data to these unused hda3 sectors (similar to what tman describes above) be simpler than trying to write to a file on a different partition?
_________________________
- Tony C
my empeg stuff

Top
#335172 - 20/07/2010 06:01 Re: App storage revisited [Re: tonyc]
peter
carpal tunnel

Registered: 13/07/2000
Posts: 4180
Loc: Cambridge, England
Originally Posted By: tonyc
Wouldn't the task of writing data to these unused hda3 sectors (similar to what tman describes above) be simpler than trying to write to a file on a different partition?

Not really, no -- reliably working out the values of n and x at runtime would be no easier than ioctl(FIBMAP), and that latter has the benefit that you can do it to a file named after your application, and be confident that your data won't get overwritten by the next developer who comes along trying the same idea.

Peter

Top
#335185 - 20/07/2010 14:55 Re: App storage revisited [Re: peter]
tonyc
carpal tunnel

Registered: 27/06/1999
Posts: 7058
Loc: Pittsburgh, PA
Yeah, that makes sense. I guess I'll give it a go. I placed an order for a new 2.5" HD, and once I do the swap, I'll be able to play around with the FIBMAP stuff. After ten minutes or so of trying to grok Mark's excellently-commented code, I think it's something I can handle.
_________________________
- Tony C
my empeg stuff

Top
#335233 - 21/07/2010 01:07 Re: App storage revisited [Re: tonyc]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
You should probably give "hdparm --fibmap xxxx" a try on the empeg first, just to make sure that FIBMAP actually works there. smile

Assuming it does, then the code you'll need is much, much simpler than what hdparm uses. Assuming your file is 1024 bytes (or less) in size, there will be only a single disk block, and it will always be present (after creating/fsyncing the file).

So no need to handle tail-packed files, "holes", or any such extravagances.

Cheers

Top
#335248 - 21/07/2010 15:36 Re: App storage revisited [Re: mlord]
tonyc
carpal tunnel

Registered: 27/06/1999
Posts: 7058
Loc: Pittsburgh, PA
Okay, that's good, though I'm probably better off just lifting the whole shebang rather than trying to slice the "extravagances" out -- I'd probably end up taking something important out, and while I don't have any use for more than 1024 bytes right now, I can't say that'll always be the case. I could always just create and write to multiple 1k files, but that seems silly if you've already done the heavy lifting to get the walk_fibmap code working for files with multiple blocks.

As an aside, in looking at the Hijack source, I see a comment that indicates that 88/128 of the empeg state bytes were used by the player as of v2.0, and then you use 16 bytes for Hijack settings. Do you know offhand if any new state bytes were eaten by settings new to 3.0 alpha? Do we pretty much have a complete map of the state block that we can rely on to implement a user ioctl for writing data to the flash?
_________________________
- Tony C
my empeg stuff

Top
#335258 - 21/07/2010 21:01 Re: App storage revisited [Re: tonyc]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14493
Loc: Canada
I don't remember if much more was used by v3 player software. Maybe a byte or two. Somewhere I have a map of (mostly) exactly what player data is stored where in the savearea.

But not with me here.

EDIT: Actually, I forgot: I _always_ have the full empeg stuff with me when I travel! smile

The attachment is a snapshot of the player's data structures from some point in v2 development. I don't think it has changed much since then.

And I doubt anyone would mind it being posted here now, after all of these years. smile

EDIT2: And before anyone asks.. no, I don't have a copy of, or access to, and have never seen, the player source code. I'd like to, but that's a different matter. wink

Cheers


Attachments
savearea.txt (565 downloads)
Description: v2beta player savearea (?)




Edited by mlord (22/07/2010 11:00)

Top
#335263 - 21/07/2010 23:47 Re: App storage revisited [Re: mlord]
tonyc
carpal tunnel

Registered: 27/06/1999
Posts: 7058
Loc: Pittsburgh, PA
Okay, that's way more detailed than what I'd come up with.

So, with the player's data ending around 0x58, and with the Hijack data beginning around 0x6e, maybe for now we can just block off the 16 bytes before the Hijack data (0x5e-0x6d) for user apps?
_________________________
- Tony C
my empeg stuff

Top
#335265 - 22/07/2010 00:56 Re: App storage revisited [Re: tonyc]
tonyc
carpal tunnel

Registered: 27/06/1999
Posts: 7058
Loc: Pittsburgh, PA
Actually, I'm seeing data in 0x60-0x63 of my save area for some reason. v3.0a11, so maybe they added more than a couple of bytes? Or maybe I've just got some junk bits in there for some reason?
_________________________
- Tony C
my empeg stuff

Top