Charles' Log: week of 4 December

  1. December (Monday). 

    DB2 Documentation:
    Marc Armstrong (marmstrong@danly.com) notes that the DB2 databases can be manipulated directly from the command line (Hurrah! GUIs are for wimps), a la: 

       (login to db2inst1)
   db2 connect to (database name, e.g. dev1)
   db2 "select (some columns) from (some table)"
      (or, in general, any standard SQL statement in quotes)
    The on-line doc (local to e250) for "db2" command line program for manipulating DB2 databases is at http://e250.ibsentg.com/db2doc/en_US/html/db2n0/.  The full doc set is at IBM DB2 Technical Library; the "db2" command line program is described in the "Command Reference" document.

    Restructuring files
    I did a massive restructuring of the net.data and static html files for Danly, in order to make it possible to view any store from within any interface.  Commerce Suite/Net.Data takes an IMHO really dumb approach to determining which macro files are used to display which categories or products -- it keeps the entire macro file path, relative to the "/stores" directory, in a field in the database.  That path includes the store directory, which is essentially the "interface directory.

    This is contrary to "Object Programming 101" -- interface and object (data, if you will) must always be kept cleanly separated. 

    I've hacked a cleaner approach to this as described in the (numbered) steps below.  The result is that a store is uniquely identified by a store number, sometimes called a merchant number (also known by different variable names, including mrn, merchant_rn, cgmenbr, prmenbr, etc.).  An interface is a set of net.data and static html files that control how the information in a store appears and is handled.  The interface directory name is passed via the variable "idn", which also takes the place of the now obsolete net.data variable storeDirectoryName.

    The information about the store number and interface directory is passed via the query string in the URL from page to page (including the "static" HTML pages as well, which aren't truly static, but use Javascript to write some of their URLs based on the query string).

    1. I've moved the static files for a particular interface into the interface directory, previously called the "store directory", e.g. /opt/WebSphere/CommerceSuite/stores/Danly1.  I've edited them to use entirely relative pathnames for images, include files, etc.

    2. For a given interface, one must also define a web-server Alias that points to the interface directory.  E.g., 
         Alias /Danly1  "/opt/WebSphere/CommerceSuite/stores/Danly1"
      This alias is used by net.data macros that must write URLs that refer to static pages, since relative URLs from the CGI-generated pages are meaningless.  (This, alas, is the only thing that keeps the whole scheme from being completely flexible wrt interface directory, although this is at least easy and doesn't require editing any interface files.)

    3. All of the net.data macro files now use the variable "idn" in place of storeDirectoryName, and do not have the interface directory name hard-coded in (which is how WebSphere Studio generates its macro files, yecch).  It is the responsibility of every net.data file and html file to include the idn and mrn (merchant number) variables in every clickable URL it generates.

    4. I have (manually) changed the database fields categsp:csdisplay and prodsgp:psdisplay to be "/macros/catdisp.d2w" and "/macros/proddisp.d2w", for all rows in the respective tables.  (It may be necessary to redo this occasionally, if more data gets imported with the "old" values.)

      To make this work properly, I've created a /opt/WebSphere/CommerceSuite/stores/macros directory, which contains very short macros that essentially redirect to the proper macro.  For example, catdisp.d2w in that directory contains: 

      %if (idn != "")
   %include  "/$(sdn)/macros/catdisp.d2w"
%else
   %include  "/Danly2/macros/catdisp.d2w"
%endif
      where /Danly2 contains the original macro files as produced by WebSphere Studio.

    IBM Tech Support
    ...noted that DB2 (as distinct from Net.Data) fixes and patches are downloadable from ftp://ftp.software.ibm.com/ps/products/db2/fixes/english-us/.  Other goodies may be visible by travelling up the tree and back down.

  2. December (Tuesday). 

    Commerce Suite Problem Resolution
    Finally was able to get through to IBM tech support re: our CommerceSuite problems. 

    1. Our problems with re-starting the CommerceSuite instance were caused by the number of merchant licenses (equivalent to number of stores, although we didn't know that previously) being set to 1.  Tech support said "just raise the number from the configuration manager" (e250.ibsentg.com:4444, "merchant licenses" tab).  Seemed curious that we could just raise that number, but I set it to "10" and now the instance starts just fine.

      The confusing part about this is that the limit is applied inconsistently.  We can create a new instance with a new database, and the limit is ignored; but if we stop a running instance and then try to restart it, the limit is applied.  Argh.

      (Tech support diagnosed the problem by having me examine the log files at /opt/WebSphere/CommerceSuite/instance/DEV/control*.log, looking at the most recently created logs.)

    2. In the meantime, we've not been able to use the WebSphere Studio store creator to publish a brand new store.  Prasanth Mulukutla at IBM tech support sent me a listing (apparently not available from their web site) of common publishing problems and solutions. 

      The section on "command execution failure" solved our problem; the database name (for our CommerceSuite instance DEV) in the configuration file /opt/WebSphere/CommerceSuite/instance/DEV/config/ncommerce.conf was wrong.  (It was "DEV6", apparently the result of something we did wrong in the multiple times we had to create new instances/databases because of the previous "can't start instance" problem!!)

    3. As a result of solving these problems, I was able to proceed to create and publish a new store Danly0, containing sample data, that works properly with the shopping cart.  The next goal is to determine, by looking at the sample store, what information (what fields in what tables) is required by the shopping cart that is currently not in our test Danly data.

  3. December (Wednesday). 

    DB2 --> CSV
    While investigating database fields for sample store Danly0, wrote a quick 'C' filter program db2csv.c to turn output from "db2" command-line queries into CSV file format, for easier perusal.  Here's a sample query that uses it.  (Also discovered that the "db2 connect" times out in relatively short order, so there is no need to QUIT or TERMINATE it.)

    Custom Shopping Carts
    Investigating the "shopping carts" (aka "interest lists")... turns out that you can't just customize the stores/storename/macros/shopcart.d2w file.  The task that displays the shopping cart (SHOPCART_DSP) has, by default, "mall" scope -- which means that the task picks up the shopping cart definition from somewhere -- I don't know where(!) -- and uses that.  From e250.ibsentg.com/ncadmin, I went in through Site manager, task management, and set SHOPCART_DSP to use "store" scope for a particular store (Danly1), and set it to use the file stores/macros/shopcart.d2w (which uses the same "idn" trick described above to load the shopcart.d2w file from the proper interface directory).

  4. December (Thursday).  Got a primitive shopping cart working.  Hurrah!

  5. December (Friday). 

    Net.Data programming
    I'm seeing some inconsistencies in how Net.Data "commands" handle form and URL data.  It appears that commands like ProductDisplay will pass/accept data passed both in the URL and as part of a METHOD=POST FORM submission.  On the other hand, ExecMacro seems to take only one or the other, so where it is called to handle a FORM submission, all of the data (that would normally be part of the URL query string) must be placed in HIDDEN fields.  Not a big deal, but potentially very confusing.