Thank you for visiting the Rackspace Community
The The Community is live! Post new content or topics so our teams can assist.

Please contact your support team if you have a question or need assistance for any Rackspace products, services, or articles.

.NET FileProvider.ListObjects(...) get more results

This question is not answered.

How can I get ore than 10,000 results using the CloudFileProvider ListObjects() function.  The 2nd OPTIONAL parameter is Limit of type INTEGER?, but if I provide it, the call just blows up and exits.  Any way to trap errors?

All Replies
  • Hey sdevoy - I'm going to move this into our developer forum and get a .NET guru to take a look at this. We'll get you an answer soon.

    Thanks,

    Alan Bush
    Technical Community Manager
    Rackspace Cloud

  • sdevoy --

    Do you have a stack trace you can provide? Or, better, some code I can peruse?

    Also; which version of the OpenStack .NET SDK are you using?

    I'll be happy to get this figured out.

    -- Don Schenck, Developer Advocate

  • Don,

    Thanks for the reply.  With some experimentation I found the MAX value allowed for "LIMIT" is 10K.  I also figured out how to use "prefix" and "marker" to limit the items returned and to return the next set of objects.

    I guess I figured out everything I asked.

    Sean

  • Sean --

    You got it.

    However, going forward ... how can I make this information more obvious? What was missing from your experience that would have made it better, easier, faster?

    Any feedback is fantastic.

    Finally, thanks for answering here; it makes it easier for folks to find an answer when searching in the future.

    All The Best,

    -- Don 

  • Hi Don,

    A single example using Windows XML load function to send native commands would be FANTASTIC.  That would open up functions like Bulk Upload and delete.

    What works BEST for me is sample code.  Some useful examples might be:

    • Listing Objects in a LARGE container.
    • All Objects (more than 10k)
    • Filter by folder (more than 10k)
    • Upload a ZIP or TAR instead of looping through 700,000 files
      aka Bulk Upload from Windows
    • Bulk Delete From Windows
    • Looping through the object list and deleting single objects is rather slow.

    Thanks for the response.

    Sean

  • Don,

    I just found from Intelli-Type that CloudFilesProvider has a BulkDelete function.  Is it documented anywhere?  I must be missing a link to some documentation.

    Sean

  • Sean --

    I don't think BulkDelete is featured in any documentation. That's not to say it's a secret or unsupported -- it just didn't get covered.

    IntelliSense should be enough to guide you along to use BulkDelete. Also, if you're adventurous, you can download the SDK source code and look through the unit tests ... you should see it covered there.

    If you want to simply clear ALL the objects in a Container, it may be easiest to delete the Container with the option to delete the contents, then turn around and create the Container again if you still need it. Not the most elegant solution, but it may be the simplest.

    Does this help?

    -- Don 

  • Sean --

    Wow ... thanks for the great suggestions.

    Regarding using an XML file to send commands: You may also want to investigate using PowerShell. There's a (very) new PowerShell module, PoshStack, that works with OpenStack. It's built on the SDK Assemblies.

    -- Don

  • I sort of stumbled on this: http://openstacknetsdk.org/docs/html/M_net_openstack_Providers_Rackspace_CloudFilesProvider_BulkDelete.htm documentation and it has been HUGELY helpful.

    I have written a routine to delete everything in a sub-folder.  I accidentally added "upload/thumbnails" and "uploads/Thumbnails", each with some 300,000 files.  Dropping all 1,000,000 files and restarting sounded dreadful.

    I am going great now.  

    Sean