SFTP Create File Fails to detect new file sometimes

[rodbland]

Hello

I have an existing flow which makes a copy of a file on dropbox and google drive when a new file is detected in an sftp folder.  It looks like this: (I have not configured any custom settings other than the name of the flow)

For the most part this works fine, but occasionally it misses a file and I can't figure out why.  It seems to be when a few new files are uploaded only a few seconds apart. Is there some way to make the new file detection process more robust?

Since the files always have different names, I'm wondering if I could take a different approach where I could simply list the files in the folder, make copies on dropbox and gdrive, and then move the files just copied to a subfolder.  That way I can be sure I'm not missing any files (I think).  If that is a better way, can someone help explain how I could set the flow up that way?

thank you, Rod

[v-yamao-msft]

Hi @rodbland,

You said that “it seems to be when a few new files are uploaded only a few seconds apart”, do you upload the file that failed to be detected directly after the previous file?

Which flow plan are you using? Please check the maximum flow frequency at here:

https://flow.microsoft.com/en-us/pricing/

Besides, the throttling limits that API calls per connection for SFTP connector is 60 seconds:

https://docs.microsoft.com/en-us/connectors/sftp/

I am afraid that it is run out a limit so that it won’t be detected by flow.

Further, it is possible to list files in the folder, then copy the files to Dropbox and Google Drive, then move the files to a subfolder using the SFTP action Copy file. I guess you would like to delete the files under the root folder, you could add an action Delete file at last.

Hope it could be a reference for you.

Best regards,

Mabel

[rodbland]

Hi Mabel

We are on the premium plan.

Per day around 100 new files are uploaded.

I dont' understand why the flow would miss a file even if it ran out of API all limit.  Wouldn't any remaining file be picked up in the next flow run?

Could you provide an example or a link to an example of how to acheive creating a flow where we list files in the folder, copy files, then delete?

thank you, Rod

[v-yamao-msft]

Hi @rodbland,

About an example on how to list files in the folder, copy files, then delete, please take the following image for a reference. In my scenario, I am trying to copy files from OneDrive for Business to Google Drive, please take it for a reference.

 

Best regards,

Mabel