SAS Portlets, Widgets, Themes and Tutorials for sale

Working with a customer where we have decided to utilise the current Oracle Data Warehouse as much as possible as the source of data for reporting, given the significant investment in the data warehouse over the last few years.

Rather than extract the Oracle data into SAS we were testing using SAS Information Maps and Web Report Studio over the top of the Oracle Star Schema’s.  I have to say I was impressed with the performance of the Web Report Studio reports given the number of layerswe were traversing, i.e

Web Report Studio > Information Map > SAS Workspace Server > Proc Sql >SAS Access to Oracle > Oracle (and back again).

Anyway one issue we found was that we couldn’t apply SAS Date Formats against the Oracle date fields, if we did it would just show ******* instead of a date.

We could use a datepart function to substring the date field and that worked ok, but that could only be applied in the Information Map, not against the Metadata definition which means other tools such as SAS OLAP Cubes and Enterprise Guide etc wouldn’t pick it up.

We tried  changing the Metadata definition for datefields from DATETIME20. to say DATE9. , but then our friends ****** would appear.  Even worse if we reapplied the DATETIME20. format to the metadata definition it set all the dates to 1960.  We had to delete the metadata definition and reimport it to get the correct dates again.

The solution was in the “Usage Note 6413: SASDATEFMT option can be defined globally in the config file or as an environment variable with SAS/ACCESS Interface to Oracle” here:

http://support.sas.com/kb/6/413.html

Although it talks about SAS 8 we applied the following statement to the sasv9.cfg file:

-set SASDATEFMT date

Rebooted the workspace server, deleted and re-imported the Oracle Table Metadata and wallah the Oracle date fields were know registered as Date. and aplpying date formats in the Metadata definition, Information Maps or Enterprise Guide worked the way they should!

As an aside in working through the options we worked out that thwre is a magic number that if you take it away from the numeric Oracle data value you get the numeric SAS date value, whihc makes sense I suppose given they both juts start counting days to manage dates, they just start counting from different points.

Great post over on Hexware’s blog  with recommendations on what you should be documenting as part of your BI / Data Warehouse project.

I know of a couple of SAS Data Warehouse sites that are using a Wiki as a collaborative environment for documentation and we are using Mediawiki on a SAS BI/DW project I am working on at the moment.

We are using the Wiki to document all our processes, development standards, business rules and content (when we get time of course).

One of the things I struggle with is how to integrate the technical metadata, that resides in the SAS Metadata Repository, with the business metadata we are storing in the Wiki.

To that end at sasInct we are developing a web service that will interogate the SAS Metadata Repository and then display the results via a Mediawiki plugin.  The goal is to enable the technical metadata to be streamed to the Wiki in realtime and remove the need to copy and paste etc, or maintain two version.

We are also starting to make use of the description fields in Information Maps to provide business metadata to users when running a report, but we haven’t come up with an approach yet of how to populate that from the Wiki, or how to enable a user to click on an object in a report and open the relevant Wiki page with the business metadata.

Feel free to comment if you have done something cool in this space.

I blogged earlier on how to purge your SAS Metadata Repository to reclaim the space from deleted metadata objects. Purging the metadata repository removes the deleted metadata objects from the repository.

However as the metadata repository is physically stored in SAS Datasets, you haven’t actually reduced the size of the metadata repository disk storage. To do this you have to run the  %OMABAKUP macro with the reorg=yes option.

I have created a three part article that outlines how to do each step (well it has two parts for now as I am still working on part three and will blog it when I have finished)

Great post by Mike Ferguson called Data Management Needs A Solid Foundation - The World of Metadata over on the Dataflux Blog

I agree with him when he says that he struggled with the definition of Metadata as “data about data”, so I really like his version of:

“When it comes to data, does the user know what it means?”

You may find that over time your SAS Metadata Repository seems to be getting bigger and bigger. The reason for this is that when you deleted an object in the Metadata Repository SAS doesn’t physically delete the object it just marks it ready for deletion.

To remove all the objects that have been flagged for deletion you need to purge the Metadata Repository. According to SAS help Purge:

“permanently removes all metadata items that have been marked for deletion in the selected repository”

You can purge the repository via SAS Management Console:

• Metadata Manager > Active Server
• Right click on the relevant Metadata Repository
• Purge

You must have security rights (authorisation) to pause the server in order to perform this task.

Be careful not to click truncate or format by mistake and of course backup your Metadata Server before performing this task.

Out of interest SAS Help defines formating your Metadata repository as:

“formats the selected repository, which removes all metadata definitions in the repository.”

and truncating as:

“Deletes all of the metadata objects in the selected repository, but does not delete the object containers or remove the repository registration”

hence the warning about ensuring you have backed up and click the correct option.