Dynamic Downloadables

ShannonShannon Member
in Bugs & Feature Requests edited May 2013
I had an integration with a photo processor that has gone out of business. I will be converting over to FoxyCart to sell downloadable versions of the photos in addition to other subscription services we offer.

One function they had in their API that I loved which might be a nice addition to FoxyCart was the ability to add a product URL for the raw image (or whatever the downloadable might be) when the item was added to the cart. These files can be stored in a location that is only available to the FoxyCart servers. What this allows if for the dynamic addition of downloadable items when the user purchases them. The value in this is that my site has thousands of photos. I only sell a very small percentage of them (probably less than 1%). It is way too much trouble to add all of the photos as downloadable products just in order to sell a few.

Does this kind of thing make sense? Even the third party apps I've seen require that you send the datafeed to them (which I'd like to continue to have sent to my servers) and/or the indexing of all the products. Does anyone else see value in this kind of feature?

I suppose one thing I could do would be to add the URL as a custom option. Then when I receive the feed I can make my software respond to the feed by submitting the form that creates the product (currently there is no function in the API that allows this). But would the links be created properly if the product didn't exist at the time of the purchase? I can also anticipate that the datafeed might be temporarily down or that the feed may take longer than the timeout interval to dynamically upload the products. What would happen in those cases? Does the transaction error out if the downloadable product doesn't exist on FoxyCarts servers?

Thanks for your thoughts and comments.

Best,
Shannon
Tagged:
Comments
  • fc_adamfc_adam FoxyCart Team
    @Shannon,

    So probably the first thing to note is that you can indeed host your downloadables where ever you want. We have an integration listed on our wiki for self-hosted downloadables that you could utilise and customise to meet your needs. Details on that here: http://wiki.foxycart.com/integration

    Could you expand a little more on what you mean by creating the product? Will you have all of your photos hosted somewhere for serving them when people purchase them?

    As a quick example, this is one way you could approach it:

    * Upload your photos to a hosting solution - the self hosted downloadables integration has a set up that works with Amazon S3, so that could be a good option.
    * On your site, I'm assuming you're listing all of your products for sale. Those add to carts would point to your FoxyCart store with a unique code for each product that is linked to a file you've uploaded.
    * When someone purchases a file, the datafeed would be sent over to your self-hosted script, grab links to the files purchased and send a custom receipt off to the customer with links to download.

    If you did need to do something custom with creating products (again, not sure what you meant there), you could process the datafeed on your end and create an automated job (like a cron) that processes the queue of jobs regularly and sends of receipts. That way, the datafeed (which does expect a response within 15 seconds) can process quickly and your time-intensive processes can run outside of that scope.
  • By "creating the product", I meant adding the product to the "Products | Downloadables" on the fly.

    I will take your suggestions and play with this a bit and let you know if I need any further guidance. Thanks for yous help.

    Shannon
  • fc_adamfc_adam FoxyCart Team
    @Shannon,

    Ah yep - I understand now. Yeah - the self hosted downloadables approach is what you'll want, and that will mean you won't actually use the native downloadables within FoxyCart at all.
  • That's what I thought. So I guess my initial suggestion still stands. FoxyCart has done a lot of work implementing downloadable products. It would be nice to not have to replicate all that work. To elaborate a little bit on my idea, here's what I had in mind.

    Consider the following link:

    <a href="https://tennisrecruiting.foxycart.com/cart?name=Ethan Gardner&code=Photo90414&category=Photo Download&quantity_max=1&price=10&url=http://www.tennisrecruiting.net/photos/album.asp?id=11165&photo=3&image=http://www.tennisrecruiting.net/photos/img/9/0/4/1/4/t3339.jpg&quot;&gt;Add to Cart</a>

    If a download with product code Photo90414 exists, then everything works fine. If not, then the cart returns an error and will not add the item to the cart.

    Now consider the following link:

    <a href="https://tennisrecruiting.foxycart.com/cart?name=Ethan Gardner&code=Photo90414&category=Photo Download&quantity_max=1&price=10&url=http://www.tennisrecruiting.net/photos/album.asp?id=11165&photo=3&image=http://www.tennisrecruiting.net/photos/img/9/0/4/1/4/t3339.jpg&download=http://photos.tennisrecruiting.net/img/9/0/4/1/4/rr400758.jpg">Add to Cart</a>

    The file located at "http://photos.tennisrecruiting.net/img/9/0/4/1/4/rr400758.jpg"; is not available to the public but is only available to the FoxyCart servers. Instead of generating an error when the downloadable product does not exist, it retrieves the download from the specified URL just as if I had submitted the form on your site. All the values are there --- name, code, price, file. FoxyCart is able to grab and cache the file specified with the image parameter. Why not do the same with a download?

    My site currently has over 84,000 photos. They are all stored in our Oracle database. Only a very small percentage will ever be purchased. So my options right now seem to be:

    1) Use FetchApp.com. Since this requires I redirect the datafeed to their servers, this is a non starter.
    2) Use self_hosted_downloads, which requires me to use PHP and MySQL and keep all the images indexed and synchronized between the two databases. Not ideal, but possible.
    3) Not use downloadable products and implement it all myself. Probably a better option that #2, but still a lot of duplication of work already done by FoxyCart.

    Consequently I just wanted to put the idea out there to see if anyone else thought it might have any value. I'll probably end up doing #3 for now, but would switch in a heartbeat if FoxyCart ever rolled out something like what is suggested here.

    Best,
    Shannon
  • fc_adamfc_adam FoxyCart Team
    @Shannon,

    Allowing you to specify custom downloadable URL's rather than having to create the product in the admin is something we've discussed in the team, and plan to add as a feature when time allows - but at this stage there are a number of other features that are higher up the list that we're working through.

    Is there any reason why you couldn't take the self hosted downloadables script and update it to point to your oracle database to fetch the images instead of the MySQL database it uses out of the box? No reason to duplicate everything. Number 3 and number 2 both don't use downloadable products (the core FC feature), so they're both similar in that regard - it's just what you use to build it that's the difference.

    Also - as a side note, you mention that fetchapp isn't an option as you'd have to point the datafeed to their servers - there is nothing stopping you from sending the datafeed to your server, and then in turn sending it over to fetchapp's server as well. There is a script on our wiki that gives you an example of how to do that: http://wiki.foxycart.com/integration/interceptor
  • fc_adamfc_adam FoxyCart Team
    @Shannon,

    One other thing to note - we're currently working on finalising an awesome new API that would actually allow you to create a downloadable product via an API call - the API will allow you to perform basically any action you currently do in the FoxyCart administration via an API call. That's a little way off yet, and we don't have a timeframe of public release - but we're very excited to get it out there.
  • Good call on the datafeed. I should have thought of that. Also I think implementing my own will be easier than modifying #2. The API will be a nice addition. Perhaps I can upload all the images when that is done. The custom downloadable URL will be ideal, though. My raw images aren't that big (a couple of 100K each). They are probably as easy to cache as product images. The link feature would save uploading 84,000 x ~300K worth of images (about 25GB) to your site when only a few hundred are ever purchased. Especially since there is no reason why they need to take up any space on your site at all (except maybe temporarily when a purchase takes place).

    Just out of curiosity, if the API did exist, I know that it says that there is no limit to the number of downloads that we can add to your site, but does that really mean 84,000 downloads?

    Also I tried to add my suggestion to the requests.foxycart.com site, but nothing happens when I click "Post a new idea..." What am I doing wrong?

  • brettbrett FoxyCart Team
    Hi @Shannon
    @fc_adam just asked me to comment on your last question, but a few quick notes first :)

    As soon as I read your first post I thought, "Oh, awesome. That's a validation of an idea we've already discussed and have planned." So, that's cool. We think it's a great idea, and I'm glad you think it'd make sense too. (Sounds perhaps like another system already did it, so that's awesome as well.)

    One thing I'm trying to think through would be a way to fake the "pass in the URL that only FoxyCart can access" with our existing functionality. Thinking out loud here, now that we have Twig as the template language, it'd be easy to pass in an extra product option (the URL), but encrypted… Then modify the templates so that specific option doesn't get output to the customer anywhere.

    You could then actually decrypt that in Twig and generate a download link right on the FoxyCart-generated receipt (web and email copies). (We'd have to add a custom decryption method to Twig, but that'd be within the realm of possibilities.) The URL generated could point to a simple script on your end that received the encrypted URL, fetched it and served it to the customer (perhaps after checking to ensure it hasn't expired or been downloaded too many times already).

    Just thinking out loud there. Lots of possibilities. But ultimately, we do plan on taking the approach you mentioned, and we've already mapped it out. Just not enough programmer-hours to make it happen yet.

    As far as 25GB worth of downloadables, we'd try to make it work, but if we're losing money on your account (and unless you're doing above average transaction volume, we probably would be) we'd probably try to work with you on a different approach. Realistically though, I think that the number of files you have would make a _lot_ more sense for you to just continue to host on your end.
    Also I tried to add my suggestion to the requests.foxycart.com site, but nothing happens when I click "Post a new idea..." What am I doing wrong?
    Odd. That's powered by UserVoice.com, which has been remarkably solid for the past few years. Are you seeing any javascript errors in your console (if you know how to access that), or perhaps is it giving you a very difficult to find error text somewhere?
  • @brett, I figured out the requests.foxycart.com problem.

    With respect to your idea, that sounds interesting. In my particular case, though, it doesn't need to even be that clever. Instead of hosting my raw photos on a server that is available to the public, I have them saved in a directory on a web server only available to a restricted set of IP's. For example, here are some publicly available links to the thumbnail and web quality photo of a player:

    http://www.tennisrecruiting.net/photos/img/8/9/9/8/0/t2610.jpg
    http://www.tennisrecruiting.net/photos/img/8/9/9/8/0/p17234.jpg

    The corresponding raw hi-resolution photo for this photo is available here:

    http://photos.tennisrecruiting.net/img/8/9/9/8/0/r402550.jpg

    Right now you can't access that file, but I can easily open this site up to the IP 204.13.109.11 (or wherever the foxycart server is). That's what I did for the other service we used. As long as the fetch of the file passes through your servers, I don't need to worry about deleting the file or anything like that. I assume the temporary links you generate will only work for a certain number of uses or for a certain time period like they do now.

    In general, though, I have no problem encrypting the parameter. It might be more general purpose or satisfy more of your customer's needs - you're (obviously) more in touch with that than I am. I'd possibly just make in an option (like HMAC encoding) in case the end user doesn't need it (like in my situation).

    I think I'm going to go ahead and post the idea to the requests site to see if it gets any votes. In the meantime, my plan is to copy the files into a publicly available folder and routinely purge them after they've been there for 48 hours. I can create a page on our site with the links and/or send a separate email with the links after the transaction takes place.

    Thanks for your help on this. Your level of support is really first rate. I am a big fan of your tools and your company.

    Best,
    Shannon









  • fc_adamfc_adam FoxyCart Team
    @Shannon,

    Thanks for posting the request and for adding your thoughts for your requirements - that's great.

    One thing to note - if you did go the available self-hosted downloadables route - you could put all of your files on Amazon S3 for not too much at all really. That way you wouldn't need to dynamically add/remove products at all - they'll all be available and it'll just work without you having to do anything.
  • @fc_adam, thanks for all your help. The storage is not the problem, really. I have them all stored on photos.tennisrecruiting.net now. Even if I went the Amazon S3 route, I would need to index the files in the self-hosted downloadables MySQL table and keep it synched. That's not too hard, but for now I think it may be even easier to just generate the links myself. We'll see. I have a lot of implementing left to do.
  • @fc_adam, @brett,

    We have a similar issue as Shannon, many ebooks for sale, but too many to manually upload to FoxyCart admin. That is why we've used the self hosted downloads integration (https://wiki.foxycart.com/integration/foxycart/self_hosted_downloads).
    We much prefer the end user experience of downloading directly through foxycart so would love to try out the api to add ebooks.

    From https://api.foxycart.com/docs it looks like we can experiment, but is there a target date for when we could go live with such a system?
  • winstonwinston FoxyCart Team
    @blac

    Right now we don't have a hard date for the API, we're focusing on one thing at a time and at the top of our priority list for the minute is 2.0 - which we're hoping to release early April.
Sign In or Register to comment.