Online Store Automation - Populating Our Store (Part 3)

Article
05/12/2006

To keep populating my store, I need a way to get lists of new titles (or previously unlisted titles) to add to my database. For this, I take advantage of the Amazon Web Services (or AWS) and the fact that for each title in their list, they keep a list of similar titles. So, as I update the stock/pricing information for each title, I can query Amazon to see if there are any similar titles to this one that I had not already covered. Here is the code that accomplishes this.

    1:                          // Get similar titles

    2:                          AmazonNodeLister lister = new AmazonNodeLister();

    3:                          List<string> similar = lister.GetSimilarProducts(curAmazonItem.Asin);

    4:                          foreach (string simitem in similar)

    5:                          {

    6:                              if ((simitem.Length == 10) && (System.Char.IsDigit(simitem[0])) && (!File.Exists(textEditData.Text + "\\" + simitem + ".dat")))

    7:                              {

    8:                                  AddStatus(" Similar = " + simitem);

9:

   10:                                  AmazonItem amazonItem = new AmazonItem(simitem);

   11:                                  file = File.Open(textEditData.Text + "\\" + simitem + ".dat", FileMode.Create);

   12:                                  formatter = (IFormatter)new BinaryFormatter();

   13:                                  formatter.Serialize(file, curAmazonItem);

   14:                                  file.Close();

15:

   16:                                  WorkItem workItem = new WorkItem();

   17:                                  workItem.Type = WorkItem.WorkType.QueryIngram;

   18:                                  workItem.Data = textEditData.Text + "\\" + simitem + ".dat";

   19:                                  workList.Enqueue(workItem);

   20:                              }

   21:                          }

22:

23:

   24:          public List<string> GetSimilarProducts(string asin)

   25:          {

   26:              SimilarityLookup lookup = new SimilarityLookup();

   27:              lookup.AssociateTag = "FOO";

   28:              lookup.AWSAccessKeyId = "BAR";

   29:              lookup.Shared = new SimilarityLookupRequest();

   30:              lookup.Shared.ItemId = new string[2];

   31:              lookup.Shared.ItemId[0] = asin;

   32:              lookup.Shared.ItemId[1] = null;

   33:              lookup.Shared.ResponseGroup = new string[3];

   34:              lookup.Shared.ResponseGroup[0] = "Request";

   35:              lookup.Shared.ResponseGroup[1] = "ItemAttributes";

   36:              lookup.Shared.ResponseGroup[2] = null;

   37:              SimilarityLookupResponse resp = service.SimilarityLookup(lookup);

38:

   39:              List<string> res = new List<string>();

   40:              if ((resp.Items != null) && (resp.Items.Length >= 1) && (resp.Items[0].Item != null))

   41:              {

   42:                  foreach (Item item in resp.Items[0].Item)

   43:                  {

   44:                      if ( (item.ItemAttributes != null) && (item.ItemAttributes.ISBN != null))

   45:                          res.Add(item.ASIN);

   46:                  }

   47:              }

48:

   49:              System.Threading.Thread.Sleep(500);

   50:              return res;

   51:          }

52:

This code essentially asks Amazon for product info on a specific ASIN (which is the Amazon SKU). In the case of a book, the ASIN is essentially the same as the ISBN number for the book. As part of the product information, I can request a list of similar products as suggested by Amazon. In this case, this is simplified by asking amazon for the list of Similar products through a "SimilarityLookup". Since similar products aren't necessarily books, I verify that the returned products do have an ISBN number assigned to them. Keep in mind that ISBN numbers also include things such as CDs and DVDs but I filter those out at some other point.

Oh and a final note on why I call sleep on the current thread at the end. Amazon restricts queries roughly to one per second from a specific IP address. I sleep for 1/2 second as I do other tasks between queries, this allows me to throttle my traffic to the AWS.

For more info on the Amazon Web Services, check out the following: https://aws.amazon.com/

Online Store Automation - Populating Our Store (Part 3)

Additional resources