FTP AM-I-STUPID?

Rad

Well-known member
So I have got to get this figured out...

We can't FTP a pkg back to the station i'm told, because it takes about 937.2 hours for minute-thirty pkg (slight exaggeration) But I see and hear of people doing this all the time. In fact, we had a crew from another station work out of our newsroom last week and FTP'd their pkg back via our internet and it took only minutes.

Is there a good primer/how-to video/ or wiki that can help me get some good working knowledge of FTP and moving these huge files?

Rad
 

micaelb

Well-known member
It all depends on the size of your file and the upload speed of whatever network you are going to use. We've found most sports arenas have very fast upload speeds and a 2:00 pkg, usually about 500 mb, takes 5 minutes, maybe 10. We use Filezilla and a Cisco VPN program to put it right into our server.
There are slow networks that take much longer.
 

b-roll

Administrator
Staff member
There definitely needs to be a little compression before your can easily FTP video back to the office. Uncompressed SD output from Avid runs about 200mb per minute of video. With good compression settings (h.264) you can get this down to 20mb/minute and that should FTP rather quickly.

Granted these are times for SD video - HD video may be different.

kev
 

zac love

Well-known member
I've FTPed from at least 3 dozen hotels & FedEx Kinkos. Only time I got better FTP speeds than when I was at my house was when I was at the DNC & RNC.

There were even times when I had to FTP things back to NY & it took less time for me to drive 3 hours home & FTP than it was to stay & find a fast WIFI near the story.

My experience was that I could transfer about 1min of video every hour of HDV (same size as DVCPro, DVCam & DV) at FedEx Kinkos which had a solid 50Kbps upload speed.

At my house, I had speeds about 4x faster, meaning I could send that same 1min of video in 15min.

When FTPing, just be very mindful of every frame you're sending. Leave off the extra heads & tails if you can.
 
I would second the h.264 compression and check your computers wireless card should be a 802.11G or N type the N is the newer standard and supposed to be many times faster when you have a router at the location that supports it. If you are accessing through wifi instead then it can be all over the place depending on how fast your service is and is supposed to be.

Now I say this with the understanding that I do not FTP news pieces but do deliver much larger videos to clients over ftp.
 

Rad

Well-known member
There definitely needs to be a little compression before your can easily FTP video back to the office. Uncompressed SD output from Avid runs about 200mb per minute of video. With good compression settings (h.264) you can get this down to 20mb/minute and that should FTP rather quickly.

Granted these are times for SD video - HD video may be different.

kev
So a 1.5 gig file, at 20mbs per seconds, would take about an hour and 15 minutes. Which is doable for non-breaking stories and with a good connection that won’t drop.

As far as I know, HD vs SD doesn't matter, it's just about file size.

My Knowledge of how FTP works is limited. If I have a great upload speed, does it matter what's going on on the receive end of things? I suspect there may be issues there.

Now, lets say I have a AVI file that's 1.5 gig. Help me as I walk through this: I use a FTP program, that program is set up to take my file (I guess I simply drag it over to the program, then uploads that to a server back at the shop with some sort of FTP receive program. is that right?
 

b-roll

Administrator
Staff member
It depends on what you're using to FTP the file. If your station sets up an FTP server in your station - then once you've uploaded the file, it is sitting in your station and ready to go rather quickly into your edit system.

If you use a "third party" FTP site, then the station would have to pull the file in from this off-site FTP server and that could take as long as your upload.

We work with a number of stations - and use many different forms of FTP. In our typical workflow...

1) edit story on Avid laptop.
2) export the story as a compressed .MOV using secret sauce of H.264 settings. (this can take 10 minutes or so for a 1:30 pkg)
3) upload this shrunken file to a centralized FTP server.
4) stations pull that file from centralized FTP.
5) stations decode compressed .MOV as it enters edit system.

In our version - there is a bit more time on the encode and decode end, but the actual FTP travel is a lot faster. Since the pushing of the file to the FTP server has the most variables (is my wireless card working well? Is somebody downloading porn at the starbucks I'm at - taking my bandwidth? Are a bunch of people in the Kinkos texting their vote to American Idol?) I like to have that be as short as possible. The rest of the processes are hardwired and take a fixed amount of time.

Hope that helps...

kev
 

dhart

Well-known member
No, it shouldn't take 937.2 to upload your story :) But it's rare to find upload speeds faster than 1.5 MB/sec. On a wireless connection expect less. Follow Kevin's instructions and you'll be okay.
 

zac love

Well-known member
No, it shouldn't take 937.2 to upload your story :) But it's rare to find upload speeds faster than 1.5 MB/sec. On a wireless connection expect less. Follow Kevin's instructions and you'll be okay.
I would be happy when I would find 200KB/sec. I think I only got things close to 1.5MB/sec was when we had a dedicated line set up at the DNC & RNC (but it was probably closer to 850-1100KB/sec). 1.5MB/sec would send a 90sec DVCPro pack in 3min, imho that is closing in on real time transfer speeds.

One thing also to be aware is that FTP is a pretty old standard, so make sure you have good software. The "resume" function can be a life saver if you're 80% done & lose your connection. Otherwise you have to start over from scratch.

Another thing that I found was useful for editors back at the shop is putting the file size into the file name. That way if they're watching it come in, they can see how much is left.
 
As the others have said, you have to compress the video. Which means that someone at the other end has to uncompress it.... so you're going to have to test to see what works at both ends.

Compression is all about the bit rate. H.264 is really good compression, meaning you can get a lot of quality at a low bit rate, which gives you small file sizes. But the more you compress, the longer it takes to do the compression.

You might be able to get useable SD quality with compression set to 1200kbps.. which should give you a minute:thirty piece under 20mb in size - which will send fast.
 

Kent

Member
Compressing program

At KMSP in Minneapolis we've started to heavily rely on Digi Delivery (similar to an FTP in a graphical shell with a dedicated hard drive next to our Avid ISIS racks) or Streambox file transfer to ship our packages *on deadline* back to the station.

We cut on Avid Newscutter laptops in the field and save the file as an 'Avid codec' MOV file. Now our wide screen 480 line MOV file package will be somewhere around 400 mb give or take. We use the free Handbrake program to compress our files, just applying their 'Apple TV' default setting. Handbrake will compress a 2 minute package in around 2 minutes (depends upon how much movement is in the package and how much 'work' the software had to crunch). Now I'll have a file which is closer to 30-50 mb in total file size. As long as one can find an internet with at least 400 kb/s upstream, that package will be back at our station in about five to ten minutes. We have ATT and Verizon USB air cards in our Streamboxs, so if we can't find a WiFi, we often can find a good G3 data connection.

If we're out of town doing a satellite shot, we're booking at least one less feed window by doing file transfers, so the money saved can easily cover the costs of the cellular air cards. In heavily used cell sites, finding free WiFi may give you better transfer speeds, but if one is out in the boonies, I often see 300 kb/s up on a G3 cellular connection.

We do an hour newscast at 5pm and do 90 minutes at 9pm with a total of about 8 to 10 locally shot and edited packages across those two newscasts. We'll cut about 2/3 to 3/4 of our packages in the field and so far we're probably at about 80 percent file transfer success every day of the week.

To be safe, I insist the reporters to get me a script a half hour earlier and set aside a half hour at the tail to get my package compressed and sent. That way if the file transfer dies, there is always that five minute window to plug into the truck satellite or microwave link and send it that way. We've found that the few artifacts we see in from our compressed file transferred package looks way better on the air than our more heavily compressed digital microwave path these days. This past week, four of the five packages I shot and edited in the field were sent via file transfer and only once did my transfer fail and had to rely on our microwave.

The key at the station is to have a dedicated box of some kind taking the files in and is on the same local LAN the Avid's are on, so transferring a file from one platform to another will only take seconds.

Our Newscutter editors simply pull that M4v file into their bin and it will take Avid a couple of minutes to covert it.

So the idea of an instant transfer isn't realistic, but we are going from hitting 'send' button in the field to actually hitting air playback typically within fifteen minutes every day of the week while on a real newscast deadline.

Kent Peterson
 

Ben Longden

Well-known member
Just a lateral thought....
My FTP uploads at 91kbps, but I can upload email at 1Mbps

Hotmail is now offering a 10Gb file attachment capability. If I used my Hotmail account, and sent the file to the office (to their Hotmail) account, then would the whole thing transfer faster - 1Mbps upload, and 20mbps download -

I was talking to the chief engineer at the station (way before the Hotmail 10Gb availability) and he was saying the FTP speed is limited to around the 91kbps...

Thoughts?

Ben
 

zac love

Well-known member
I was talking to the chief engineer at the station (way before the Hotmail 10Gb availability) and he was saying the FTP speed is limited to around the 91kbps...

Thoughts?

Ben
I've gotten a 4.2MB/s FTP upload, hells-yeah for a research University's data pipe! But that maybe is SFTP, which could be different than the original FTP?

http://www.boutell.com/newfaq/definitions/ftp.html

I don't have time to really read into the differences, but there could be a limit. Just keep in mind that you've got a ton of places where the transfer rate could get bottle necked, & the rate is only going to be as fast as the slowest gate.

Still 91KB/s is about 40% faster than what I'd get at Kinko's, & average for most hotels I've FTP'ed from. My cable modem is still the fastest I've ever gotten outside of massive dedicated data lines.

I still don't get why FedEx or other business support businesses don't offer more high speed uploading by the day / hour for us type guys on the road.
 

focusthis

Well-known member
FTP n00b here: The Quicktime player on our new Edius desktops wouldn't recognize the .mov file sent to us. Could someone please point me towards a solution? This is way more difficult to troubleshoot than putting the yellow tape into the blue tape player!
 

zac love

Well-known member
FTP n00b here: The Quicktime player on our new Edius desktops wouldn't recognize the .mov file sent to us. Could someone please point me towards a solution? This is way more difficult to troubleshoot than putting the yellow tape into the blue tape player!
Well there are a dozen things that could go wrong.

The most obvious is that your version of QuickTime doesn't have the right plug-ins or codecs to play the .mov file. ".mov" is just a wrapper & the contents can be any flavor of digital video.

If you can open the file when it isn't FTP'ed then the problem is probably that whoever sent the file, sent it as ASCII instead of Binary, but I doubt that is the case, b/c most "drag & drop" FTP software these days don't make that become trouble. If they were doing FTP through a command prompt, then maybe they forgot to change the settings & sent the file wrong.
 

Ben Longden

Well-known member
I still don't get why FedEx or other business support businesses don't offer more high speed uploading by the day / hour for us type guys on the road.
Amen to that.

Three hour drive to the gig, 40min shoot (half of that waiting for the fog to lift) then three hour drive back to the office, and then cut and upload (to save a further two hours travel time)....

Hah... I should have just cut it on the laptop and driven the five hours to the bureau... FTP was as slow as a wet week.

But to my boss' credit, the story ran that night..

Ben
 

tvwcyj

Active member
I Ftp everyday from a landline connection I get about 500-1MB/kbps. I also feed back stories through my Droid's Wifi or a Wifi Hotspot when in the field. I get about
50-150/kbps on those connections. I use Premiere and Final Cut to edit and use their software encoders for h.264 at about 3mbps. I have never had a file go over 60 MB. You should be able to feed that within 10 mins at the most.
 
Top