Unoffical empeg BBS

Quick Links: Empeg FAQ | RioCar.Org | Hijack | BigDisk Builder | jEmplode | emphatic
Repairs: Repairs

Topic Options
#67051 - 04/02/2002 22:46 Emplode 2.0-b7 Too CPU Bound on Sync
grgcombs
addict

Registered: 03/07/2001
Posts: 663
Loc: Dallas, TX
I don't know if you'd classify this as a bug or not, but when syncing from an SMB mounted drive, Emplode uses 100% of the CPU and almost none of the network bandwidth. I figured it'd be just the opposite.

I'm using XP, everything over 100BT switch (except empeg on 10BT, of course). It's an 800mhz P3.

Just seems odd that it would need so much cpu for what would seem to be a primarily network based operation.

Also, we need a cancel on syncing ... though I doubt this is possible. It seems crappy that I have to kill the app whenever anything bad goes wrong while syncing up.

g
_________________________

Top
#67052 - 05/02/2002 02:03 Re: Emplode 2.0-b7 Too CPU Bound on Sync [Re: grgcombs]
Shonky
pooh-bah

Registered: 12/01/2002
Posts: 2009
Loc: Brisbane, Australia
I originally was copying files from our server to my RioCar like this :

Server 100BT -> 100BT Switch -> 100BT card in my PC -> back out same 100BT card -> same switch -> 10BT RioCar

I too was getting basically 100% CPU. So I copied a large number to a local drive and synce from there. Same deal 100% CPU. Didn't seem to be a huge (if any at all) speed increase).

Just did a 260Meg transfer from my local drive in about 600 seconds =~ 430kb/sec

Using Bulletproof FTP thru Hijack I was getting around 700kb/sec. Any suggestions to get the speed up?

Running Win98 + 2.0b7 + Hijack v180.

FWIW using the Hijack FTP client I get pretty much spot on 500kb/sec (as longer as the player isn't playing) through a 10BT crossover cable at home.
_________________________
Christian
#40104192 120Gb (no longer in my E36 M3, won't fit the E46 M3)

Top
#67053 - 05/02/2002 03:21 Re: Emplode 2.0-b7 Too CPU Bound on Sync [Re: grgcombs]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5683
Loc: London, UK
The protocol is riddled with latency issues. These can eat CPU without using much network bandwidth. With Ethernet, the situation is improved, but there's still latency in certain places.

Also, did you have tracing turned on? It'd have warned you at startup if you had, so you probably didn't. If you did, there's a shedload of data going to disk, and that can introduce a bunch of CPU usage -- every line needs to be sprintf'ed, which eats CPU.
_________________________
-- roger

Top
#67054 - 05/02/2002 03:24 Re: Emplode 2.0-b7 Too CPU Bound on Sync [Re: Roger]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5683
Loc: London, UK
Anyway, I'll see if I can get the source profiler working, and see if there's anywhere obvious that emplode is spending all of its time.
_________________________
-- roger

Top
#67055 - 05/02/2002 11:24 Re: Emplode 2.0-b7 Too CPU Bound on Sync [Re: Roger]
grgcombs
addict

Registered: 03/07/2001
Posts: 663
Loc: Dallas, TX
Just as a quick comparison, JEmplode used comparitively little CPU time and was limited by the network throughput of SMB it seems.

Greg
_________________________

Top
#67056 - 04/03/2002 12:58 Re: Emplode 2.0-b7 Too CPU Bound on Sync [Re: Roger]
Kit
journeyman

Registered: 03/10/2000
Posts: 69
Loc: San Diego, CA US
2.0b11 with Emplode under WindowsXP uses 100% CPU while performing an fsck after hitting the max mount count. I am running Hijack v226 so just to be sure I will test with the original kernel.

On another note, any chance of emptool ever getting the code that fixes corrupt database issues? I gave up on emptool + linux after frequently getting into states that only emplode would fix. (This is generally after a sync fails, only emplode can rebuild the database without crashing.)

-Kit

Top
#67057 - 04/03/2002 16:44 Re: Emplode 2.0-b7 Too CPU Bound on Sync [Re: Kit]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5683
Loc: London, UK
...100% CPU while performing an fsck...

Yeah, I noticed that myself today. I'll take a quick look. Medium term, I plan on getting the source profiler to work, which should give me some hotspots to look at.

any chance of emptool ever getting the code that fixes corrupt database issues?

I don't know. If you have specific things that can cause it to crash, report them in the Bug Reports forum. They'll get looked at.
_________________________
-- roger

Top
#67058 - 17/06/2002 06:15 Re: Emplode 2.0-b7 Too CPU Bound on Sync [Re: Roger]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5683
Loc: London, UK
which should give me some hotspots to look at.

Had a quick look at this today, while attempting to lift some code for another project.

It's not CPU-bound -- it's sorta-kinda busy-waiting for a background thread. This will affect the speed, but not appreciably. I'll see if I can figure a way to remove the busy-loop. Don't hold your breath, though.

_________________________
-- roger

Top
#67059 - 17/06/2002 08:23 Re: Emplode 2.0-b7 Too CPU Bound on Sync [Re: Roger]
SE_Sport_Driver
carpal tunnel

Registered: 05/01/2001
Posts: 4903
Loc: Detroit, MI USA
Thanks Roger - you rock!
_________________________
Brad B.

Top
#67060 - 18/06/2002 07:41 Re: Emplode 2.0-b7 Too CPU Bound on Sync [Re: SE_Sport_Driver]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5683
Loc: London, UK
FITNR.

If anybody cares for the technical details, check out http://www.differentpla.net/~roger/devel/background_threads/, in particular background #3 (what emplode used to do) and background #4 (what it does now).



Edited by Roger (18/06/2002 08:36)
_________________________
-- roger

Top
#67061 - 18/06/2002 08:11 Re: Emplode 2.0-b7 Too CPU Bound on Sync [Re: Roger]
SE_Sport_Driver
carpal tunnel

Registered: 05/01/2001
Posts: 4903
Loc: Detroit, MI USA
The link isn't working right now?
_________________________
Brad B.

Top
#67062 - 18/06/2002 08:32 Re: Emplode 2.0-b7 Too CPU Bound on Sync [Re: SE_Sport_Driver]
peter
carpal tunnel

Registered: 13/07/2000
Posts: 4180
Loc: Cambridge, England
The link isn't working right now?

Roger is some kind of muppet and has linked to our intranet webserver?

Peter

Top
#67063 - 18/06/2002 08:37 Re: Emplode 2.0-b7 Too CPU Bound on Sync [Re: peter]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5683
Loc: London, UK
Yeah, what he said. Fixed.
_________________________
-- roger

Top
#67064 - 18/06/2002 09:21 Re: Emplode 2.0-b7 Too CPU Bound on Sync [Re: Roger]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31597
Loc: Seattle, WA
Fascinating article, Roger. Gonna have to remember that one for when I need it later.
_________________________
Tony Fabris

Top
#67065 - 18/06/2002 09:30 Re: Emplode 2.0-b7 Too CPU Bound on Sync [Re: Roger]
peter
carpal tunnel

Registered: 13/07/2000
Posts: 4180
Loc: Cambridge, England
http://www.differentpla.net/~roger/devel/background_threads

3a. The observers called from the background thread notify an event, and the foreground thread's message loop sits in MsgWaitForMultipleObjects?

AFAICS, this is just like 3 but without the busy-waiting. Or have I missed something?

Peter

Top
#67066 - 19/06/2002 06:36 Re: Emplode 2.0-b7 Too CPU Bound on Sync [Re: peter]
Roger
carpal tunnel

Registered: 18/01/2000
Posts: 5683
Loc: London, UK
No, not really.
_________________________
-- roger

Top