Google Search  |  Meridian Systems  |  Meridian Streaming  |  Restaurant  |  WiKi  |  Duncan's Meridian Info  |  Board Rules  |  Restaurant Rules
Page 1 of 2 1 2 >
Topic Options
Rate This Topic
#271913 - 2018-01-16 21:48 Export speed guess?
Registered: 2017-12-23
Posts: 26
HalJayGreene Offline
Harmless
HalJayGreene Offline
Harmless

Registered: 2017-12-23
Posts: 26
Loc: Denver, CO, USA
Hi All,

Quick question:

I did two complete backups of my Sooloos DB (one from a Mac, one from Windows), which shows around 9k albums. Each took about 3 days. Whew!

Now I'm trying to do a full FLAC export of everything in anticipation for switching over to Roon. I started with the Windows version (Control:PC) but it kept hanging after a few thousand tracks, so I switched to the Mac version (Control:Mac). After 24 hours, about 15k tracks have been transferred to a late-model QNAP NAS, hard-wired through a TP-Link switch to a Mac router. At this rate I guess the complete export will take about a week, based upon a track count of 121k.

Is this a reasonable rate of transfer? Why are transfer of tracks so much slower than backups? Is it because of the NAS? Both backups were done to USB drives connected directly to the computer. Is there a faster method?


Thanks,


Hal

Top
#271942 - 2018-01-17 16:12 Re: Export speed guess? [Re: HalJayGreene]
Registered: 2002-02-14
Posts: 8,393
ncpl Online sleepy
President of the Imperial Galactic Government
ncpl Online sleepy

President of the Imperial Galactic Government

Registered: 2002-02-14
Posts: 8,393
Loc: Surrey, UK
Hi Hal,

same as I said on the Roon forum wink

IMHO exporting across the network (to a NAS) was a very slow process. My conclusion was that the tagtool.exe (which within Sooloos tags the untagged Sooloos music files to become regular tagged files) does NOT work across network locations.

If you re-run the export to a local drive (e.g. your USB drive) then it should run significantly faster.


Rgds,
Nick

#15SE
Top
#271943 - 2018-01-17 16:18 Re: Export speed guess? [Re: ncpl]
Registered: 2010-12-09
Posts: 3,204
Ratbert Offline
Hooloovoo
Ratbert Offline
Hooloovoo

Registered: 2010-12-09
Posts: 3,204
Loc: Europe
+1, and ensure the PC or Mac is connected via Ethernet not wireless as that also made a significant performance difference.


.
Top
#271944 - 2018-01-17 16:25 Re: Export speed guess? [Re: ncpl]
Registered: 2017-12-23
Posts: 26
HalJayGreene Offline
Harmless
HalJayGreene Offline
Harmless

Registered: 2017-12-23
Posts: 26
Loc: Denver, CO, USA
So what happens to the files when they are exported across a network? Will there be something defective about them re: tagtool.exe error? Should I just cancel this export (three days and 45k files running) and switch over to a connected drive?

Top
#271945 - 2018-01-17 16:56 Re: Export speed guess? [Re: HalJayGreene]
Registered: 2004-12-09
Posts: 6,191
ChrisLayerUK Offline
President of the Imperial Galactic Government
ChrisLayerUK Offline
President of the Imperial Galactic Government

Registered: 2004-12-09
Posts: 6,191
Loc: Colchester, Essex, UK
I would expect the network to be slower.


No 18 in the Hall of fame smile I have simple tastes; I am content to settle for the best: GIK monster bass panels, rear wall.
MC200 500GB, QNAP TVS 471i3. Roon into 2 MS200's. Explorer 2. Bluesound Pulse 2 and Mini. 596 v1.89.2, 518, G61R, M60.2 Rosewood, M33C, M33 Rears, M1500 Sub, Foxsat HD, Sony Blu Ray, Pioneer 5090, 2 Chromecast's for LRB videos.
Top
#271946 - 2018-01-17 16:58 Re: Export speed guess? [Re: HalJayGreene]
Registered: 2002-02-14
Posts: 8,393
ncpl Online sleepy
President of the Imperial Galactic Government
ncpl Online sleepy

President of the Imperial Galactic Government

Registered: 2002-02-14
Posts: 8,393
Loc: Surrey, UK
Nothing as far as I recall is wrong with them. It is just that something about tagtool.exe across a network drive doesn't work well and it takes a stupidly long time.

Cancel that export and start a new one on a local drive. Your library won't take too long. I think at USB3.0 speeds you should see an album per minute.


Rgds,
Nick

#15SE
Top
#272012 - 2018-01-19 14:49 Re: Export speed guess? [Re: HalJayGreene]
Registered: 2017-12-23
Posts: 26
HalJayGreene Offline
Harmless
HalJayGreene Offline
Harmless

Registered: 2017-12-23
Posts: 26
Loc: Denver, CO, USA
I started an export on the same drive as the Sooloos DB. The drive is 6TB and the Sooloos DB takes up less than half of that. I expected the export to fit in the other half, but after only 2/3 progress the export failed due to a full target disk.

Is the export significantly larger than the DB? How can I guesstimate the size of the export?

Top
#272015 - 2018-01-19 15:35 Re: Export speed guess? [Re: HalJayGreene]
Registered: 2002-02-14
Posts: 8,393
ncpl Online sleepy
President of the Imperial Galactic Government
ncpl Online sleepy

President of the Imperial Galactic Government

Registered: 2002-02-14
Posts: 8,393
Loc: Surrey, UK
The export will be similar in size to the music2 folder in your Sooloos store. That is where the files are.

Also, what does your Sooloos summary page, see Sooloos Library Stats tell you?




Rgds,
Nick

#15SE
Top
#272016 - 2018-01-19 15:40 Re: Export speed guess? [Re: ncpl]
Registered: 2008-05-23
Posts: 7,606
Carl Offline
President of the Imperial Galactic Government
Carl Offline


President of the Imperial Galactic Government

Registered: 2008-05-23
Posts: 7,606
Loc: Central England, UK
Not overly significant but worth noting ... in Sooloos there is only one copy of the album artwork for each album, when exported the artwork is embedded into each album track file.


861v6+ID40, HD621, 8000.2, 7200HC, 7200, SKYQ Silver, PS3, PDP-LX6090, QNAP TS870-Pro (8*6TB), Roon, iPadAir2.
[In Reserve] C15, MD600 (2*3TB), TwinStore RM (2*2TB)
Top
#272021 - 2018-01-19 19:00 Re: Export speed guess? [Re: HalJayGreene]
Registered: 2014-07-14
Posts: 19
ScotAV Offline
Harmless
ScotAV Offline
Harmless

Registered: 2014-07-14
Posts: 19
Loc: Central Scotland
I recently rebuilt my system and am running the Roon trial. I performed a Native Export of my Sooloos library and expected this to be more or less the same size as my existing library approx. 1TB, some 3500 albums.

To my surprise the Native Export size was approximately 50% larger than my Sooloos library at around 1.5TB. I don't know the reason for this but others may know the answer? There are no duplicate albums in the export library and it shows more or less the same amount of 3500 albums. This ties up with what you have experienced carrying out your export.

I know this may not be of any help to you but at least you know that I have experienced the same issue.


Scotav
Top
#272040 - 2018-01-19 22:50 Re: Export speed guess? [Re: ScotAV]
Registered: 2017-12-23
Posts: 26
HalJayGreene Offline
Harmless
HalJayGreene Offline
Harmless

Registered: 2017-12-23
Posts: 26
Loc: Denver, CO, USA
This is what I see:

Top
#272065 - 2018-01-20 09:42 Re: Export speed guess? [Re: HalJayGreene]
Registered: 2002-02-14
Posts: 8,393
ncpl Online sleepy
President of the Imperial Galactic Government
ncpl Online sleepy

President of the Imperial Galactic Government

Registered: 2002-02-14
Posts: 8,393
Loc: Surrey, UK
This tells you that you have approx 2.7TB of library on Sooloos.

The export will be not less than this.

Take Carl’s point about artwork and estimate that 122,000 tracks will embed a cover art of whatever size your typical artwork is.

That should give you a ball park figure.

Take from the stats thread I linked above and multiply your 9000 albums by an average of maybe 350mb or 400mb per album

This would suggest between 3TB and 4TB as well.

Does this match what you found so far?


Rgds,
Nick

#15SE
Top
#272067 - 2018-01-20 10:00 Re: Export speed guess? [Re: HalJayGreene]
Registered: 2014-07-14
Posts: 19
ScotAV Offline
Harmless
ScotAV Offline
Harmless

Registered: 2014-07-14
Posts: 19
Loc: Central Scotland
I agree with ncpl. From what I have experienced you have roughly 2.7TB of existing Sooloos files, add 50% to that = 1.35TB giving a total of 4.05TB (Sooloos Native Export) plus your existing 2.7TB = 6.75TB.

There's obviously no way both Sooloos and the Native Export will fit on your 6TB drive and does roughly equate to your 2/3 disk full statement. I'm afraid it's time for a rethink.

This is only based on my experience but does tie up with ncpl's reply.


Scotav
Top
#272087 - 2018-01-20 13:48 Re: Export speed guess? [Re: ncpl]
Registered: 2017-12-23
Posts: 26
HalJayGreene Offline
Harmless
HalJayGreene Offline
Harmless

Registered: 2017-12-23
Posts: 26
Loc: Denver, CO, USA
I can tell you anecdotally (I am re-doing the export now to a larger disk) that the last export failed at around 2/3 and 3TB (which maxed out the disk), which suggests to me that the full export will occupy around 4TB (significantly larger than the Sooloos DB).

I'm exporting now into 8TB so it should finish in a day or two (fingers crossed). I'm mildly distressed that the export to the NAS (very, very slow) at least gave me an accurate running account of the progress, while the export to the attached T-Bolt server just gives me a blank screen. But when I watch the export folder I can see albums being added continuously so I know it's working. Maybe the screen will refresh at the end? I'd like to know if there are any "failed" exports.

Related Q: Will Roon be happy with the way Control:PC is organizing the file/folders? There don't seem to be any options for changing this.

Thanks,
Hal

Top
#272090 - 2018-01-20 14:11 Re: Export speed guess? [Re: HalJayGreene]
Registered: 2014-07-14
Posts: 19
ScotAV Offline
Harmless
ScotAV Offline
Harmless

Registered: 2014-07-14
Posts: 19
Loc: Central Scotland
Hi Hal, yes the whole process is painfully slow but such are the joys of computer software/hardware, mines took around 24 hours or so. I had no failed exports at the end of the process from Sooloos.

Roon seems to be more than happy handling the Native Export library with no issues so far. Whilst Sooloos is very good software Roon is in a different league. I can only speak for the new version 1.4 as this was already implemented when I started the trial. It seems that they have taken on board a substantial amount of their users requests. Their forums are a great way of gaining knowledge of how the system works as is this forum.

I would be interested to know your final results regards export size? Best of luck with your transfer.


Scotav
Top
#272093 - 2018-01-20 15:03 Re: Export speed guess? [Re: ScotAV]
Registered: 2002-02-14
Posts: 8,393
ncpl Online sleepy
President of the Imperial Galactic Government
ncpl Online sleepy

President of the Imperial Galactic Government

Registered: 2002-02-14
Posts: 8,393
Loc: Surrey, UK
Will Roon handle exported albums? Yes most of the time.

It will vary when you look at multi CD sets. Sooloos never really showed these as whole albums whereas Roon will show the discs as one album. I.e. correctly.

When you compare Sooloos discs to Roon albums you will need to keep this in mind. Always compare track counts to be sure.

Roon has many features that will help you identify and manage the ingested library.


Rgds,
Nick

#15SE
Top
#272096 - 2018-01-20 15:35 Re: Export speed guess? [Re: ncpl]
Registered: 2017-12-23
Posts: 26
HalJayGreene Offline
Harmless
HalJayGreene Offline
Harmless

Registered: 2017-12-23
Posts: 26
Loc: Denver, CO, USA
An aside: By the time I have my export library in place I will be managing the whole shebang with a Nucleus+. Yay! I will def let everyone know how that goes.

Top
#272154 - 2018-01-21 18:37 Re: Export speed guess? [Re: HalJayGreene]
Registered: 2017-12-23
Posts: 26
HalJayGreene Offline
Harmless
HalJayGreene Offline
Harmless

Registered: 2017-12-23
Posts: 26
Loc: Denver, CO, USA
K, weird. As mentioned before, a full native export from a Sooloos DB of ~2.7TB to a late-model QNAP NAS with ~3TB free space failed after 5 days, 2/3 of the way through with a "disk full" error. A re-start of the process to an attached T-Bolt disk finished after less than 24 hours with no failed exports and the correct # of tracks, but only took up 2.4TB space. Hmmm...

Top
#272157 - 2018-01-21 22:26 Re: Export speed guess? [Re: HalJayGreene]
Registered: 2014-07-14
Posts: 19
ScotAV Offline
Harmless
ScotAV Offline
Harmless

Registered: 2014-07-14
Posts: 19
Loc: Central Scotland
Bizzare!!


Scotav
Top
#272291 - 2018-01-25 19:23 Re: Export speed guess? [Re: HalJayGreene]
Registered: 2014-07-14
Posts: 19
ScotAV Offline
Harmless
ScotAV Offline
Harmless

Registered: 2014-07-14
Posts: 19
Loc: Central Scotland
Hal, Nick, following Hals final results I thought it wise to investigate my Native Export files again. Prior to my export I created a Shared Folder on my QNAP, I also checked all the settings prior to creating and unticked the option to create a recycle bin. Foolishly I didn’t check that the Bin hadn’t been created as per my preferred options. On checking my Native Export folder I noticed that a Recycle Bin had been created within the shared folder which in normal circumstances would not have been a problem. When I checked the folder options again I found that the Recycle Bin option was still unticked, I have no idea why it was created. On checking the contents of the Bin I found that it contained some 22960 files which equated to 588 GB (approx. 60% of my existing Sooloos Library). I don't know why these files were there.

I decided to delete the existing Native Export folder, created a new folder and this time checked that a Recycle Bin had not been created! I have just finished a new Native Export the total size being 968 GB which compares favourably to my Sooloos Music2 folder @ 966 GB. I’ve yet to confirm that the Artists, Albums and Tracks are correct.

I thought this may of interest to anyone moving to Roon in the future. Anyway all is well now and looking forward to getting back to listening to my music.


Scotav
Top
#272312 - 2018-01-26 08:24 Re: Export speed guess? [Re: ScotAV]
Registered: 2002-02-14
Posts: 8,393
ncpl Online sleepy
President of the Imperial Galactic Government
ncpl Online sleepy

President of the Imperial Galactic Government

Registered: 2002-02-14
Posts: 8,393
Loc: Surrey, UK
Thanks for the follow up. That all sounds much more sensible.


Rgds,
Nick

#15SE
Top
Page 1 of 2 1 2 >

Moderator:  Carl, Duncs, ncpl 
Who's Online
7 registered (DDD, Avii, Njal, Paul Lane, ncpl, Bobfett), 37 Guests and 3 Spiders online.
Key: Admin, Global Mod, Mod
Newest Members
Southall1998, Nate, Sting, SaxMan, Kenken777
4589 Registered Users
Top Posters (30 Days)
CMB Integrations - Bruce 56
Nstzya 47
Carl 41
Ian 40
Tarik 37
Top Posters
VirusKiller 12270
Fiddler 8668
ncpl 8393
Carl 7606
Ian 7390
Forum Stats
4,589 Registered Members
17 Forums
24,457 Topics
251,253 Posts

Most users ever online: 211 @ 2015-05-05 14:40
Meridian  |  Media Centre  |  Support  |  Firmware Release Notes  |  RSS Systems  |  RSS Streaming  |  RSS Restaurant