The Foxy forums are on the move!

We're in the process of moving our forums over to a new system, and so these forums are now read-only.
If you have a question about your store in the meantime, please don't hesitate to reach out to us via email.

Some minor datafeed issues

oskayoskay Member
in Bugs & Feature Requests edited February 2010
The re-feed feature is fantastic-- we can actually test our datafeed processing with correctly-formatted data from the current version of FoxyCart.

Two little issues with the datafeed & refeed:
1) When we re-feed a transaction, it seems to always process twice. This isn't the case with normal transactions.

2) When we get a transaction processed through the datafeed, the transaction date (transaction_date) seems to be off from the actual order time by an inconsistent amount. Are there unset server times that are causing this?
Comments
  • lukeluke FoxyCart Team
    Thanks for posting, oskay, we'll definitely take a look. Because it's a new feature, there may be a couple interesting things we discover here and there.

    1) When you say it "processes twice" do you mean your script does things twice or do you actually get the data sent to your server twice by FoxyCart?

    2) That should be pulling from the database, but there might be some other stuff going on. What value are you comparing it to? Do you have any pattern to it that will help us reproduce the problem? Are subscriptions involved?
  • oskayoskay Member
    edited February 2010
    1) It appears that the data is actually sent twice by FoxyCart in this case; asking for a re-feed seems to always feed twice immediately. When an order is processed FoxyCart sends the feeds once, and our test feeds also only generate one feed. I can't see any other explanation.

    2) I believe that when a customer checks out at our site, the transaction is processed immediately-- that is, before the receipt screen.

    At that moment, our payment gateway sends us confirmation that they've received funds, FoxyCart sends order confirmation e-mail, and FoxyCart sends the datafeed which we process into a PDF packing slip that is also e-mailed to us at the same time. All of these generally happen within the span of a few seconds-- and I take that moment as my reference time.

    In the packing slip, we record the order date by just printing the "transaction_date" value, and it doesn't always agree well with the apparent order time.

    Surveying a few, the transaction_date indicates that the transaction actually occurred sometime in the past, by:
    6 minutes
    1 minute
    3 minutes
    2 hours & 8 minutes
    8 minutes
    10 minutes,
    11 hours -- spanning a date line
    (a bunch of more short ones)
    8.8 hours,
    etc.

    And no subscriptions are involved.
  • lukeluke FoxyCart Team
    Ok, so if i'm hearing you correctly #1 and #2 are unrelated, is that correct? The datafeed reports the same transaction_date regardless of if it's "refed" or not, correct?

    In our system the transaction date is set to NOW() as soon as the order is completed so I'm not sure how that could be off by such a wide range... very strange. As for when it's off by multiple hours, are you going off of your gateway confirmation time (as that might be specific to the timezone of the purchaser)?
  • >if i'm hearing you correctly #1 and #2 are unrelated, is that correct?

    Yes.

    >The datafeed reports the same transaction_date regardless of if it's "refed" or not, correct?

    I certainly presume so-- haven't checked that!

    >As for when it's off by multiple hours, are you going off of your gateway confirmation time (as that might be specific to the timezone of the purchaser)?

    All I'm doing is comparing the transaction_date field, as provided in the FoxyCart datafeed, to the date & time written on the FoxyCart order confirmation e-mail.
    (I presume that transaction_date is given by the system time of your servers, and has nothing to do with the gateway confirmation time.)
  • >>The datafeed reports the same transaction_date regardless of if it's "refed" or not, correct?
    >
    >I certainly presume so-- haven't checked that!

    Turns out that it's *not* the same when it's re-fed. This gets more interesting.

    * We had a transaction processed at 5:42:31 PM today, judging by the date on the email.
    * The XML that we received at the time of the order gave a transaction_date of 17:36:38 (5:36:38 PM) -- about six minutes off.
    * Looking at our FoxyCart transaction report now, I see that the order is listed there with a time of 05:42:28 PM. That's surprising, since I would expect the transaction_date to match the time stored in your database.
    * Re-feeding the transaction, the XML feed (well, both XML feeds... since we get two...) now shows a transaction_date of 17:42:28. It changed!


    Also: this order was from australia-- doesn't look like a time zone issue.
  • lukeluke FoxyCart Team
    I might be having a Monday moment here... but isn't 17:42:28 military time for 5:42:28 PM?

    I'm working on some other stuff at the moment, but I plan to run some tests locally on this to figure out what's going on. As for the time that the email shows up or is time stamped, I wouldn't put much into that AT ALL since email can be all over the place depending on how many servers it gets bounced through, if the server queue is full and tells our server to hold up, etc, etc, etc. There can be a number of reasons for the emails to show up later.
  • >but isn't 17:42:28 military time for 5:42:28 PM?

    Yes, of course. I was assuming that we both knew this.


    Let me summarize what I know about the problem: the FoxyCart XML datafeed includes an incorrect value for "transaction_date". This value is off by an inconsistent amount, that seems to vary (in my quick survey) from about 1 minute to about 12 hours. The incorrect transaction_date is only fed during the datafeed at the time of the order. Re-feeding the XML at a later time gives a value that is consistent with when the order was actually placed.


    >I wouldn't put much into that AT ALL since email can be all over the place

    That's exactly my point. I'm TRYING to use the value that FoxyCart provides for transaction date (rather than the time stamp on the e-mail) as the value that goes into our record. But I can't rely on it, because it doesn't seem to be connected to reality.

    We are not seeing an issue of delayed e-mail, nor of servers filling up and stuff bouncing around-- that's a red herring. We have very clear evidence of what's going on from the different date stamps involved in the transaction.

    The following things happen within a few seconds of each other:
    * FoxyCart sends order confirmation E-mail
    * Our gateway sends payment confirmation e-mail
    * FoxyCart sends the XML datafeed
    * FoxyCart records the transaction (including the correct time) in our transaction report.

    Together, those four time-stamped things happening simultaneously put a fairly good fix on when the transaction occurred. The only part of this that does not make sense is that the transaction_date field sent in the XML datafeed does not agree with any of those four time stamps. Instead it's off by anywhere from one minute to twelve hours.


    In the specific example that I described above, the time stamps recorded are:
    * FoxyCart sends order confirmation E-mail : 5:42:28
    * Our gateway sends payment confirmation e-mail: 5:42:53
    * FoxyCart sends the XML datafeed: 5:42:31 (really, a few seconds before this-- this is when our XML script e-mails us).
    * FoxyCart records the transaction (including the correct time) in our transaction report: 5:42:28

    These all point to the transaction happening at about 5:42:28, while our gateway takes a slow-but-plausible 25 seconds to send us e-mail. These are all self consistent. The only part of this that does not make sense is the date sent in that first XML transaction: 5:36:38; about six minutes before the transaction. But it could just have easily been about 12 hours, so it seems.
  • lukeluke FoxyCart Team
    Yes, definitely a Monday moment. You referenced email timestamps a few times and for some reason when you said it changed, I thought you were referring to the time in the transaction report (05:42:28 PM) and what you see in the refed XML: 17:42:28. With what you just clarified, I think I know what's going on. We store the data in the database when the order is completed but the object in memory doesn't appear to be updated with the correct time and, in some cases, may still show the time that the cart session was first created.

    Thanks again for your detailed response, sorry I was a bit slow there. If the fix isn't too invasive (and I don't think it will be) we can probably push out a fix for 060 on this one.
  • Great! :D
  • lukeluke FoxyCart Team
    Ok, #2 should now be fixed in 060. Thanks for bringing this up as I think its been an issue with our system for quite some time and the issue was usually reported to us in regards to timezone differences and such, but it probably had more to do with the fact that the transaction in memory didn't match what was being saved to the database. It should now match.

    As for #1... that one is very strange. I was able to reproduce it, but I'm not really sure how to fix it. It might actually be related to the flash without style bug we're working on. When it happens, the admin may be reloading twice. If you click around the admin a bit and then refeed the transaction, it seems to feed once just fine. If, however, you go into the transaction history right after completing a transaction, then it seems to feed twice. I haven't been able to lock it down completely or consistently. We'll keep looking into it though.
  • Wow-- thanks!

    As for #1, I hadn't thought of the refresh issue-- I thought that it was merely cosmetic. However, I am using a browser that produces that "flash of unstyled content," so that does seem like more than a coincidence.
  • brettbrett FoxyCart Team
    Wait, is anybody seeing a FOUC in the admin? That'd be a new one, if so. Luke, you're saying the admin is double loading for you?
  • lukeluke FoxyCart Team
    It's not super obvious, but it did seem to be a slight delay with a white screen before it loaded up the results and when it did that, I got two feeds written to my xml file. When it loaded the page directly right away, I only got one. I have this ticketed as #1049
Sign In or Register to comment.