mr ailsbury

The Photography Thread

1314 posts in this topic

Has anyone got any experience with storage and processing of RAW/jpegs on a professional photography level?

My mother in law has set up as a photographer full time and whilst she has no issues with the taking and editing of the images she has the filing system organisation of a pebble.

I built her a new pc to do the editing on (her old vista box was beyond incapable for the job) and she says it's great with regards to the speed. However when it comes to the work she has / is working on it's running into terrabytes of storage. She has a 240gb ssd, a 2tb sshd, (2x 4tb hdd that someone else put in because she thought she had run out of space) and 2x 3tb external usb drives. Unfortunately due to her not having a clue what she is doing she has duplicated/triplicated all sorts of shit in an attempt to do backups, she had filled the C:\ drive up, paniced that it was going to self detonate and copied a load of stuff over to another drive. However she moved all sorts of stuff that she shouldn't and now the machine is pretty f**ked - I am going over there boxing day for a couple of days to rebuild it again.

What is the general method of working with large volumes of raw files?

I think the following is about right:-

  1. Unload the raw files to a working folder on the ssd.
  2. Edit the raw files in lightroom whilst on the ssd.
  3. Export various jpeg formats to a job folder on a storage drive once done editing.
  4. Copy edited RAW files to job folder on storage drive and delete the working folder on the ssd.
  5. When client is happy with job burn 2 copies of the job folder to dvd, store one in the office and one off site.
  6. Carry out incremental backups (using appropriate software) onto an external drive one at a time. Again store one off site.

Does that sound reasonable? I am going to have to write a flowchart for her to follow the EXACT steps she has to follow?

In reality her portfolio to date is about 700gb, she has 10tb of storage and she has managed to fill it with copies of copies of copies :(

Edited by forteh

Share this post


Link to post
Share on other sites

Interesting to see one in bits, my girlfriend's one's been giving her a bit of grief recently so need to investigate things a bit more there.

Speaking of film, few from the summer;

tumblr_ndhs0xSI6T1staa0ko1_1280.jpg

Wiiiinneeeer. Really like this one dude!

I've had the chance to venture outside the studio and shoot some photos of Ali recently - they're all up in hurrrr.

Share this post


Link to post
Share on other sites

I would recommend building like a 12tb raid drive that can just sit near the computer full time for backups, or work out a way of sending stuff to it wirelessly and stick it somewhere else.

On the backup drive create a Rushes folder and a completed projects folder.

Rushes folder.

As soon as she gets in from a shoot, card in the reader, copy the whole file structure to the back up drive/raid into a clearly dated/named folder without doing anything to it at all (the Dcim folder and misc folder), put the dcim/miscs into card 1, card 2 etc sub folders to keep them organised.

e.g. Trials Forum party 22_12_14 > Card 1 > DCIM/MISC folders. Seperate by camera too if she's using a couple of different bodies.

then copy the whole thing onto your working drive and put them through lightroom, create an OUT folder within the working folder for all finished exports. Once all your work is done copy the whole thing back onto the Raid into the Completed Projects folder.

Then once every 6 months you can empty your Raid backup off onto a different hard drive and stick it somewhere for storage and start the process again with a clean drive.

Thats how we do it at the office I work at, but our backup drive is like an 800tb network storage thing cause we go through huge amounts of space with video work. Its a good system though if you just downsize it to work with your mum in laws needs.

It's good to keep the original files in case anything major needs to be changed after completion, and its good to have the completed work to hand because clients are super happy if you can just say yep hear you go and send stuff to them straight away.

Has anyone got any experience with storage and processing of RAW/jpegs on a professional photography level?

My mother in law has set up as a photographer full time and whilst she has no issues with the taking and editing of the images she has the filing system organisation of a pebble.

I built her a new pc to do the editing on (her old vista box was beyond incapable for the job) and she says it's great with regards to the speed. However when it comes to the work she has / is working on it's running into terrabytes of storage. She has a 240gb ssd, a 2tb sshd, (2x 4tb hdd that someone else put in because she thought she had run out of space) and 2x 3tb external usb drives. Unfortunately due to her not having a clue what she is doing she has duplicated/triplicated all sorts of shit in an attempt to do backups, she had filled the C:\ drive up, paniced that it was going to self detonate and copied a load of stuff over to another drive. However she moved all sorts of stuff that she shouldn't and now the machine is pretty f**ked - I am going over there boxing day for a couple of days to rebuild it again.

What is the general method of working with large volumes of raw files?

I think the following is about right:-

  1. Unload the raw files to a working folder on the ssd.
  2. Edit the raw files in lightroom whilst on the ssd.
  3. Export various jpeg formats to a job folder on a storage drive once done editing.
  4. Copy edited RAW files to job folder on storage drive and delete the working folder on the ssd.
  5. When client is happy with job burn 2 copies of the job folder to dvd, store one in the office and one off site.
  6. Carry out incremental backups (using appropriate software) onto an external drive one at a time. Again store one off site.

Does that sound reasonable? I am going to have to write a flowchart for her to follow the EXACT steps she has to follow?

In reality her portfolio to date is about 700gb, she has 10tb of storage and she has managed to fill it with copies of copies of copies :(

Edited by mr ailsbury

Share this post


Link to post
Share on other sites

Interesting to see one in bits, my girlfriend's one's been giving her a bit of grief recently so need to investigate things a bit more there.

Speaking of film, few from the summer;

tumblr_ndhs0xSI6T1staa0ko1_1280.jpg

Really great capture...

Wiiiinneeeer. Really like this one dude!

I've had the chance to venture outside the studio and shoot some photos of Ali recently - they're all up in hurrrr.

Mark they're awesome, really well lit too. The ones in the woods are particularly good in my opinion, also it's a less usual place to see a bike like the Inspired.

As for me, I finally went full frame. I bought a used Nikon D3 which was going cheap (although I received it to find the sensor pretty filthy, so there goes £40...), and then a 24-70mm f/2.8, since they're about 25% cheaper in Asia than they are in the UK.

Share this post


Link to post
Share on other sites

Lots of stuff!

That's awesome thanks, really useful ideas. I think what I will do is set the two 4tb in a mirrored raid and use that for the main storage, the 2tb sshd for general documents and stuff and the ssd for working copy. I'll set the external disks to be used for the incremental backups and hopefully she won't get past the 3tb mark :)

I will create a default folder structure for her projects storage that includes separate folders for originals and various exports.

Do you think she should be saving both original and edited versions of the raw files or is it possible to revert the edited to original state if need be?

Adding a remote storage isn't really feasible at the moment and optical media storage is probably more economic ( besides which it's less digital filing for her to make a mess of in the near future! :D )

Share this post


Link to post
Share on other sites

This is incredible.

1 person likes this

Share this post


Link to post
Share on other sites

That's awesome Josh, seven and a half minutes of why I love living where I do. So many Tors there I recognise and twice as many I don't.

1 person likes this

Share this post


Link to post
Share on other sites

Indeed mate. As if I ever needed reminding why I love Devon so much anyway!

Share this post


Link to post
Share on other sites

Urban Outfitters are starting to push Lomography, so there is lots of really cool camera apparel in store at the moment.

I picked up this cracking shirt today for a tenner...

10712949_1576285955919913_23154719193675

Just as a heads up it anybody wants to do a little sale shopping.

Share this post


Link to post
Share on other sites

That's awesome thanks, really useful ideas. I think what I will do is set the two 4tb in a mirrored raid and use that for the main storage, the 2tb sshd for general documents and stuff and the ssd for working copy. I'll set the external disks to be used for the incremental backups and hopefully she won't get past the 3tb mark :)

I will create a default folder structure for her projects storage that includes separate folders for originals and various exports.

Do you think she should be saving both original and edited versions of the raw files or is it possible to revert the edited to original state if need be?

Adding a remote storage isn't really feasible at the moment and optical media storage is probably more economic ( besides which it's less digital filing for her to make a mess of in the near future! :D )

This could depend on what she is using to edit with.

-We do a shoot.

-Transfer to laptop on the way home.

-Once home, put the raws into a lightroom catalog and back up to a g-drive which is daisy chained to another which automatically backs up the raws/catalog.

-One drive stays at mine, the other goes to photography partners house.

-We use lightroom so you work on the raws within a catalog which can be closed and re-opened, the edits are always there as long as you don't delete the catalog. Once edited, we update the catalog file onto the drives, no need to transfer raws again as they are always there.

-Exports jpegs for couples, back these up.

-We delete the raws and catalog off laptops and the job is done.

-6 months later, delete the raws keeping the jpegs.

Share this post


Link to post
Share on other sites

I have my own camera gear for the first time since I started! I have been lucky enough to have been handed down a canon 7d with a load of lenses as they won't fit my Dad's new 1D. Going to purchase canons new 24mm pancake lens soon.

Does anybody have any recommendations for a macbook hard drive? I was a cheapskate and bought the macbook with the smallest internal hard drive.

Here's a few of my recent shots, I know I need to shoot something other than riding.

p5pb11748597.jpg

p5pb11748599.jpg

p5pb11753507.jpg

1 person likes this

Share this post


Link to post
Share on other sites

8mZ9J0.jpg

This is taken on the causeway just outside of my village.

Edited by Little Yoshi
2 people like this

Share this post


Link to post
Share on other sites

This is my local pub.gxyir9.jpg

3 people like this

Share this post


Link to post
Share on other sites

After 4 days of sorting the mother in laws photos, I think it's in a position where by she can work properly!

After panicking that she had run out of space she had someone put a pair of 4tb drives in, however they put in 5900 rpm drives, certainly a lot slower than the 7200rpm sshd that I had put in for her to work from (I'll also point out that they hadn't screwed the disks in, just stuffed them into the case!).

So now with 16tb of space ( 2tb sshd, 2x4tb hdd, 2x3tb usb hdd) and no sense of filing she was saving raw files onto one drive, created a new lr catalogue on another, saved exported jpegs elsewhere and had no naming or folder structure. Added to this she thought duplicating raw files constituted backing up the whole lot was a mess. The 2tb drive had 200gb free, one 4tb had 500gb used, the other 4tb was being used as a backup destination for the first (set up by the muppet who installed them, but completely ignoring the hundreds of gbs of photos) on the 2tb. One 3tb drive was completely full and the other had a couple of hundred gig used in Lightroom backups.

Every single file and folder was sorted through, deleting duplicates and condensing down to one location. The total volume of her work? 1.2tb!

Mounted the 2x4tb drives in a striped raid and moved the photos + Lightroom catalogue onto there, left the 2tb for documents and set up the two external drives in media rotation to do daily backups of the whole system.

I wrote her 5 pages of instructions on how to manage the file system and deal with the Lightroom catalogue, I just hope to hell that she follows them!

Share this post


Link to post
Share on other sites

DrEvil your photos are always freakin' awesome, I'm not excited by weddings and yet your photos make me interested.

I have bought myself a cheap SLR (D3200) after my Pen got stolen and have already bought some remote slave thingies and i'm about to order a few flashes. I need a bag to store this stuff in as my rucksack isn't sufficient! I used to have a rather nice Lowepro bag which was about £70 but i'm not looking to spend that much on something I won't use very regularly.

I've seen these 2 on eBay that look to be sufficient for my needs:

This one lacks a tripod strap but I could easily sew something on

http://www.ebay.co.uk/itm/Camera-Backpack-Rucksack-Sling-Pack-Bag-for-DSLR-Black-with-free-Raincover-/321249071187?pt=UK_Camera_Cases_Bags_Covers&hash=item4acbefd053

This one has a tripod strap for a few quid more and appears to have slightly better build quality judging by the photos too?

http://www.ebay.co.uk/itm/BlackTravel-Camera-Backpack-Oxford-Case-Bag-For-Canon-Nikon-Sony-DSLR-Rain-Cover-/251736970844?pt=UK_Camera_Cases_Bags_Covers&hash=item3a9cb1565c

EDIT:

This appears to be a copy of the Lowepro I had...

http://www.ebay.co.uk/itm/7dayshop-Photographers-Backpack-Rucksack-Camera-and-Laptop-Version-Black-/361145052205?pt=UK_Camera_Cases_Bags_Covers&hash=item5415ec2c2d

Any recommendations would be appreciated!

Edited by CurtisRider

Share this post


Link to post
Share on other sites

I picked this up as a gift for a friend - http://www.henrycarroll.co.uk/books/

Gave it to them for Christmas, then gave it back to me today and asked me if I'd read it.

I hadn't, so I did.

f**k me, that is the best guide to the basics of how to take a picture I have ever read. Hugely recommended to anybody on here, I think it's available online too.

Share this post


Link to post
Share on other sites

There are a million fake Lowepro bags out there, although that one is slightly cheaper here: http://www.7dayshop.com/products/7dayshop-photographers-rucksack-camera-and-laptop-version-black-SLRPROBAG1

Saying that, the quality of the materials is shocking to say the least. Lowepro don't use great materials as it is, so you can imagine what the rip-offs are like to cut costs.

I'd recommend scouring eBay for a second hand Crumpler or Kata bag. Both those are built like tanks and go for decent prices on eBay.

Edited by Mike Deere

Share this post


Link to post
Share on other sites

This happened the other day by accident, thought it was pretty cool

post-36867-0-27881300-1421007921_thumb.j

Share this post


Link to post
Share on other sites

Few recent ones from me.

16247671695_c83ddb6383_c.jpg

15627796143_7c79664bcb_c.jpg

16245827961_fd07ffaa7f_c.jpg

16227411121_7df7baf0c4_c.jpg

16017176916_c6be78c4d6_c.jpg

2 people like this

Share this post


Link to post
Share on other sites

kYOwxb.jpg

Working away during the weeks so playing about with macro shots.

1 person likes this

Share this post


Link to post
Share on other sites

This took a lot of effort to get done, with the wind chill it must have been -10 being exposed up on stanage edge.

p5pb11814163.jpg

6 people like this

Share this post


Link to post
Share on other sites

Worth it.

Share this post


Link to post
Share on other sites

That's cool!

Awesome job

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.