John,
Thanks for the reply! Comments inlined.
--On Friday, May 20, 2005 9:26 AM -0400 John Stoffel john.stoffel@taec.toshiba.com wrote:
Alex> Nightly, an application creates a fairly large number of files, Alex> makes a new directory on an R200, and then dumps the data to Alex> it.
I assume it actually works the other way? Creates a directory on the R200 and dumps the files into there? Something like /vol/foo/nightly.<date>/... ?
It actually first creates the data locally on a server (multiple servers create their own data in fact) and only after finishing copies the data over. It's probably fairly irrelevant though. The only reason it's created locally first is because it's faster and the added redundancy makes us more comfortable.
Or you can grow the existing volume... do you need to keep all the older data you've written online?
Yes, I need to keep all the old data.
Softlinks? Can you change the application and how it creates the base directory? What happens if the directory already exists when it runs? If nothing bad, then softlinks should do the trick, otherwise you'll have to migrate data.
Yes, I can change the way the app works since the part of the process that copies the data is a separate script. Are you suggesting to make /vol/foo essentially a collection of links to foo1, foo2, etc.? That's doable, I guess.
It all depends on how fast it's filling up and how much control you have over the apps/clients.
Right now, it's about 250GB per week, but it's slowly accelerating. As I mentioned above, the app that copies the data over is just a script that I can modify. The clients don't really care either as long as they can get to the data through /vol/foo.
Thanks!
Alex.