Murmur of a webmaster

DOCTYPE Mobile

2012-01-25T08:22:00.001-08:00

It took more than a day...

I was implementing mobile view of our internet-facing web site, which is on MS SharePoint 2010.

The page’s computed width gets 980px. But I could not find out where it comes from.

We had design company prepared their designs in HTML. So my task should not have been much more than a copy and paste. However, SP pages generated based on my mobile master page (basically of copy and paste) gets its computed width as 980px. Consequently, the contents gets all too small to read, while the HTML given by the design company shows nicely on mobiles.

Towards the end of a fruitless day, I noticed that the DOCTYPE definitions are different.

I did this mobile master page, based on that for PCs, which contains those SP controls and all. Its DOCTYPE is the usual:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

and I did not think I need to change this.

One defined in the design company’s HTML is:

<!DOCTYPE html PUBLIC "-//WAPFORUM//DTD XHTML Mobile 1.0//EN" "http://www.wapforum.org/DTD/xhtml-mobile10.dtd">

With it, my SP pages look now OK on mobiles.

I am not an authority or anything. Do not know anything about the wapforum. I am writing this here just hoping that it may help you there tearing your hairs, not understanding where the hell the 980px width is coming from.

Migrate FrontPage web from IIS6 to IIS7.5

2011-08-31T00:45:00.001-07:00

After weeks of try and error, I think luckily I have archived what I wanted. Not 100% but to an acceptable level.

100% is to copy everything. Users would not even notice that it is moved. But that was not possible. I could not figure out how after all. The thing missing is user permission. That, I think I would re-setup manually. At least the admins. Then ask them to deal with their colleagues.

First, I write about the method that did not work for our site after all. I believe that is due to something particular with our site, which I could not figure out. I may have achieved the 100% if I could.

You copy the site to your new 2008 R2 server using msdeploy tool. http://learn.iis.net/page.aspx/427/migrate-a-web-site-from-iis-60-to-iis-7/

You activate (“extend”) FPSE (FrontPage Server Extension) on the copied site.

At this point, you realize that the FP user permission are missing. This is what you should do: http://blogs.msdn.com/b/robert_mcmurray/archive/2009/10/12/migrating-fpse-roles-between-servers.aspx.

In case your site is simple enough, this is the end of the story. You have achieved the 100%. However, with our site, the server health check repair did not work. It ends with an exception, without telling me what it does not like.

Giving up to copy the permission, I uninstall FPSE once then re-extend. Failed too. Stuck...

Here is the workaround that I found. I cannot activate FPSE on the fully copied site. Let us then have an empty FPSE enabled site first, and then pour the actual site data into it.

I created a blank web site, activate FPSE, and finally copy files from the old server with the following command.

msdeploy -verb:sync -source:metakey=lm/w3svc/1551257136 -dest:metakey=lm/w3svc/2,computername=newserver -skip:objectName=dirPath,absolutePath=.+_vti.+ -skip:objectName=metakey,absolutePath=.+vti.+ -skip:objectName=metaKey,absolutePath="fpexedll.dll"

The “-skip” parts are my invention. No documentation found. They are to not to copy FP related stuff. It looks work. All webs are copied, seem working. I can create FP web on them.

No real alternative yet for FrontPage Server Extension

2011-04-14T02:49:00.001-07:00

After days of frustrating analysis, my conclusion at the moment is to stay with the now-has-become-a-third-party-product FPSE.

My requirements are:

I, the server administrator, do not want to do much. I do not want to be called to intervene each time a web site is to be created. Once I have appointed developer A for admin of web site host/siteA, I do not want to have to intervene for host/siteA/siteA1 etc. Moreover, when he has got a colleague who assumes the same role as him, he could do the necessaries.
Operations for developers such as managing the web site permission, deploy their codes, should be straightforward.

With FPSE on a 2003 box, it has been near perfect. Except for one thing. No means that developers can verify if a folder is setup as “application”. They can turn a folder into an application, but cannot verify.

The story started when we start thinking of upgrading our web server, currently IIS6 on 2003 box, to IIS7.5 on 2008 R2. Soon I came to know that FPSE is no more supported. There is one as a third party product, but not from MS. Not yet for IIS7.5 but it is said to become available soon.

What is then the MS’s alternative? None that I can see.

Visual Studio 2010 supports the followings for code deployment.

FPSE
FTP
Network share
Web Deploy # Only for Web Application Projects
WebDAV # Some say you could, by mapping it as (or like) a network drive.

I do not know if you agree. But to me, 2 and 3 seem going back to the stone age.

4. Web Deploy should be the first choice. It is new so must be better. But some say that converting a web site project to web application is not as straightforward as you may think. In addition, it does not fulfill one of my requirements above. The server administrator has to do whole a lot!!

If I setup the “Application” host/siteA for developer A, he could autonomously do the child application host/siteA/siteA1, by importing the siteA1 application with the IIS7 Remote Administration. (I so far fail to do the same with VS though) But he cannot give the permission to his colleagues by himself. It is understandable. When the server administrator setup an application, he needs to not only configure IIS Manager Permission but folder permission as well, while with FPSE you do all these in one go.

Finally, 5. WebDAV too appears to have the same shortcoming, lacks the permission management delegation capability that we want. It is said “The IIS7.* WebDAV extension module supports per-URL authoring rules, allowing administrators to specify custom WebDAV security settings on a per-URL basis.” But as it reads, it is for (server) administrators only. When you connect to a site or an application, with the IIS7 Remote Administration, you do not have the WebDAV Authoring Rules icon. And to connect to the server (then you have the icon), you need to be the administrator...

And above all, needing to map WebDAV folders for publishing web sites from VS does not sound to me an optimum solution...

CSS delivered as Content-type: application/octet-stream

2011-02-25T05:50:00.001-08:00

I do not know if this could ever be any benefit of anybody. It could be just that there is something wrong with our servers…

On our site, sometimes (it appears) CSS files are delivered to clients as Content-type set to application/octet-stream.

The story starts as a few colleagues report pages are displayed ugly without styles on Firefox and Chrome. OK on IE.

I found that the CSS is stored in the browser's cache as application/octet-stream. This is tricky because once you clear the cache and go to the page again, the CSS would be downloaded OK.

There seems nobody having similar issue, or at least reporting to the net. I tried to come up with an own workaround.

I wrote an HttpModule to verify if responses for css files are generated correctly as Content-type: text/css, and correct it if not. It writes into the log file each time it find such a problematic response. By the way, in the course of writing the module and testing it, I found that IE applies styles even if it receives the css file as application/octet-stream. This is why the problem has been reported happening only on Firefox and Chrome. Furthermore, for some css, those that SharePoint generates, you can change the Content-type (to application/octet-stream for the testing) in the MIME type configuration of IIS. However, those of our own creation, those in the /Style library folder, come as text/css, even after the change in the IIS’s MIME section. So it is SharePoint that controls it.

Towards the end of the first day after the module is plugged in, the module reported that it did the Content-type correction. And a lot! We have two Web Front End in a farm. Both connect to one single Content database. Each web server generates those mal formed css responses at different timings.

Each has a IISRESET scheduled job. At different times of a day, obviously. Each generates the problematic responses after the IISRESET and stops doing so when the application pool gets recycled. (it gets recycled when the memory usage reaches the threshold)

Sh*t!! I should have requested pages from the server while the module there was reporting the problem!! I could have verified that my Content-type correction does the trick. No worries. It would come again, I guess…

Post from Windows Live Writer

2011-02-25T03:00:00.001-08:00

Let us see how it goes…

ASP:Menu SkipLink

2010-12-20T05:47:00.000-08:00

For those who landed here looking for a solution together with the cristal clear explanation, sorry, no, I do not have one :-(

Symptom.
We have migrated our internet facing site from SharePoint 2007 to 2010. We (needed to) have redone our custom master page. We did so having the 2010 minimal masterpage http://code.msdn.microsoft.com/odcSP14StarterMaster as the basis.
Then we found the Global Navigation menu, which uses the SharePoint:AspMenu control (I think it behaves pretty much the same as ASP:Menu), appears a little lower than it used to.
It does so to me everywhere. I have IE8 and FF3.5 on my PC. But later realized that it is OK on their old versions; IE7 and FF1.5.

The only difference is with the so-called "skip navigation link". It is visible (and thus takes some space) with the new master.

It was invisible with the old master.

The above screenshots are from FireBug. But up to now, I do not know what makes it "visible" or "ïnvisible".

My solution for the time being is to set DOCTYPE of the masterpage back to XHTML 1.0 Transitional. Do not ask me why...

The above mentioned minimal masterpage comes as XHTML 1.0 Strict. It is pity that I had to go back "Transitional". But this is the only workaround that I so far found.

Install Perl Modules

2010-12-14T01:24:00.000-08:00

This is really my personal memo.
I have installed a perl module for the first time in my life and actually learned a lot, so just like to keep what I did here for future reference.

The command is as simple as this. There is a switch -MCPAN for the executable itself to add a module. I log the output. Again this is just for my reference.
# perl -MCPAN -e "install Time::Period" >install.perl.time.period.log 2>&1

Actually, this failed.
It was unable to find the package, Period-1.20.tar.gz, at (in our environment) http://cpan.mirror.fr/.
I found the file at http://search.cpan.org/CPAN/authors/id/P/PR/PRYAN/Period-1.20.tar.gz. So how can I change the host (that perl goes to get packages)? You do as follows.

# perl -MCPAN -e shell <- go into the shell mode
cpan shell -- CPAN exploration and modules installation (v1.61)
ReadLine support available (try 'install Bundle::CPAN')

cpan> o conf urllist <- show the currect host list
urllist
http://cpan.mirror.fr
http://cpan.mirror.fr/
Type 'o conf' to view configuration edit options

cpan> o conf urllist pop
cpan> o conf urllist pop <- do this twice to remove the unwanted hosts

cpan> o conf urllist push http://search.cpan.org/CPAN/
cpan> o conf urllist push http://cpan.mirror.fr <- not really care but just liked to keep what were there
cpan> o conf urllist push http://cpan.mirror.fr/

cpan> o conf urllist
urllist
http://search.cpan.org/CPAN/ <- now perl tries this site first
http://cpan.mirror.fr
http://cpan.mirror.fr/
Type 'o conf' to view configuration edit options

cpan> o conf commit
commit: wrote /usr/lib/perl5/5.8.0/CPAN/Config.pm

cpan> quit

Try the first command again, and GOTCHA!

EventLog.WriteEntry from a ASP.NET page

2010-10-27T00:56:00.000-07:00

The page is run as NETWORK SERVICE. I would not ask you to either change it to LOCAL SYSTEM, or put NETWORK SERVICE to the administrator group.
The later sounds quite stupid. But a colleague of mine did find somebody proposing it as a solution on the net.

The simplest code fragment that you would find at many places on the net should look like the following. I too found it on the net.

protected void Page_Load(object sender, EventArgs e)
{
// Create the source, if it does not already exist.
if (!EventLog.SourceExists("hoge"))
{
//An event log source should not be created and immediately used.
//There is a latency time to enable the source, it should be created
//prior to executing the application that uses the source.
//Execute this sample a second time to use the new source.
EventLog.CreateEventSource("hoge", "Application");
// The source is created. Exit the application to allow it to be registered.
return;
}

// Create an EventLog instance and assign its source.
EventLog myLog = new EventLog();
myLog.Source = "hoge";

// Write an informational entry to the event log.
myLog.WriteEntry("hello world.");
return;
}

This would not run as NETWORK SERVICE. However, there is actually only one instruction that cannot be performed with NETWORK SERVICE privilege. It is the “CreateEventSource()”. The instruction to actually write an event, WriteEntry(), runs with no problem.
Since CreateEventSource() needs to run only once, you either take it out and run in a separate console app, or runs the above code only once as LOCAL SYSTEM then change it back to NETWORK SERVICE immediately after that.

[MOSS 2007 public facing site] access to MS Office documents in Document Library

2010-04-28T08:17:00.000-07:00

I am less and less certain if this is a right product…
This is THE most incredible and discouraging experience that I ever had with this product, which I have been working on to maintain our public internet site.

First of all, MANY MANY MANY thanks to the author of this blog post.
http://www.theblackknightsings.com/RemoveLoginBoxWhenAnonymousUsersDownloadOfficeDocumentFromSharePointSite.aspx
# BTW, the design is very funny. I thought it was a genuine error screen at the beginning.

The explanation is very clear, and so is the code. So I went straight plugging it in to our production site.
# I have at least compiled the DLL myself, and did a bit of test though.

We know that this product i.e. SharePoint is now widely used, especially with intranets, sites for collaboration and so on, where you know the users, they are all authenticated. However, for public facing internet sites such as ours where majority of users are anonymous, there are points overlooked, including ones as obvious as this one. And we users by ourselves need to apply workarounds… Hope with the coming 2010, it comes as a WCM product fully ready for public sites.

A day with video files

2010-04-06T14:13:00.000-07:00

Nothing to do with work. Some video stuff.

Many probably knows about .fll (FLV Flash Video) file. I have got some movie files where there are some annoying discrepancy between image and sound.

I was bored today so tried to do something with it.

There is this famous Avidemux freeware with which you could fix this discrepancy. The latest is 5.2. It appears not yet nicely support the H264 video format.

When it finds that your file is in the format, it popups:

H.264 detected. If the file is using B-frames as reference it can lead to a crash or stuttering. Avidemux can use another mode which is safe but YOU WILL LOSE FRAME ACCURACY. Do you want to use that mode?

I am not at all an expert in this area, do not know what this means exactly, so went on accepting use of the "safe" mode. That appears fine. However, if you do not do what you want at this first time you open the file and close it. The popup would never come again and as the consequence, you can not open the file, unless you go to its conf file (in XML) and manually clear the history. This was the first difficulty that I encountered.

The second thing is the sound. To adjust the sound so it goes nicely, side-by-side with the video, of course you want to see the video together with the sound in its preview mode. I use a mac at home by the way. And the latest mac version of the freeware does not seem to allow me to listen to the sound.

I went back to a previous version, 4.4. Its handling of the sound device seems ok. Good! But then it does not yet support the audio format of my files!!!

So finally what I did was the following. I open each of those files with the latest, 5.2, change the audio data format to mp3, then turn to the 4.4 and shift the starting point of the sound.

A nice holiday activity.

media="print" and proxy

2010-04-01T15:13:00.000-07:00

Really strange thing happened.

One of our developer tried to do a print-friendly rendering of a page. That is, when you print the page, logo is removed, hyperlinks is followed by their URLs written etc. A usual stuff.

He did so using media="print" trick with CSS.

He made it work nicely with FireFox but not on IE. Still needs some improvement, to support various browsers. That's OK. Not my business.

The weird thing is that it behaves differently when you get the page, css everything via a proxy.

The simplest example. If I start Fiddler to see if all come from where they are supposed to, you see it already different in the print preview...

You would normally suspect that some element of the page such as a css file fails to come, due to some authentication/authorization issue for instance, when you go thru the proxy. But Fiddler reports none. The key, linked as media="print", css seems downloaded OK. If the elements are the same, whether or not via proxy, it should behave in the same way...

After almost one day of fruitless investigation, given up... Asked him to revise it, so it works OK with IE as well, and with or without a proxy.

Mmmm...

ASP.NET Menu for Arabic i.e. right-to-left pages

2010-02-11T08:55:00.000-08:00

Just to share, or for myself to remember this small piece of info.

http://www.microsoft.com/middleeast/msdn/Asp2_menu.aspx

It is becoming more and more that the key to be a good programmer is the ability to come up with a good set of keywords for the problem in hand, to type into a search engine...

In this case... what did I type?
At first, I typed something like "ASP.NET Menu right to left", whcih did not give me any useful pages.
Then tried (already started thinking of moving to CSS adapter. No, do not want to do that. Would be quite a work...) "ASP.NET Menu Arabic". Then, Voila! I found the above.

For those who speak Arabic, it must be a well-known trick. But for us...

"The site with the id {GUID} could not be found" and BLOBCache

2009-10-27T09:55:00.000-07:00

My own memo. Nothing I myself found.
Anyway it was really hard to figure out. But I was lucky that it happened in our testing environment. So I did not have to start crying yet.

The problem was the following.
I was getting “The site with the id {GUID} could not be found” error to get some css files from Style Library. But only on the extended web application, and futhermore, only when a user logs in to the site. # Our site accepts anonymous visitors too.

I re- extended the web app, re-restored the data from our production site. Nothing helped. # I did not have the problem on the production site.

As always, I went to the net for some help, a hint at least. But this time I did not think I would find anything helpful. I was believing that the content database might have been corrupted somehow, or something…

However, there are people who had the exact same problem, and have overcome it.
http://blogs.msdn.com/joshuag/archive/2008/05/22/filenotfoundexception-the-site-with-the-id-guid-could-not-be-found.aspx

This was a big help. I did not even think about the cache.
Unfortunately, however, the solution did not work for me.

But I know now all I need to know is how to clear the cache. And found this http://blogs.pointbridge.com/Blogs/monnette_jeff/Pages/Post.aspx?_ID=15.

More (seemingly) in depth explanation can be found at http://sharepointinterface.com/2009/06/18/we-drift-deeper-into-the-sound-as-the-flush-comes/.
I did not have anough patience to read it throughly though…

In short, the UI does not work for a farm like ours, and the stsadm solution (http://msdn.microsoft.com/en-us/library/aa622758.aspx) stinks.
We have only two front-ends. So I went for the manual flush.

Accessibility : Accessibility Kit for Sharepoint

2009-07-21T06:00:00.000-07:00

There is this tool (? add-on?) exists. But after all it did not help much.

You find a couple of things in the package but we were only interested in what they call HCCE (HiSoftware Compliant Code Engine) and WebZonePart controladapter.

The HCCE is nothing more than a custom base class for Page Layout, and what it does is simple string replacements.
For instance, MOSS Publishing feature generates <SCRIPT> in its output HTML, which violates XHTML. It has to be <script type=”text/javascript”>.
It does this, the replacement in the output. But only if you specify it, exactly as above, one by one, in its config file.
So it is not like, it does the work if you say you want to “comply” to XHTML, or HTML. “Compliant Code Engine”… Funny..

And the other cr*p, WebZonePart controladapter, it is a controladapter. So as you know, it replaces with table with div.
By default, MOSS generates lines like below for each webpart on the page. Of course it all violates standard.
<td id="MSOZoneCell_WebPartWPQ3" orientation="Vertical" name="MSOZoneCell" relatedWebPart="WebPartWPQ3"…

The adapter replaces those with divs. Sounds nice. But my problem was that it does not produces title and border of the part even if you configure you want them.
Luckly, the package contains the source (not for HCCE. HCCE comes only as binary (of debug build…)). So I managed to have them back.

What still remains not OK are those XHTML not compatible codes generated mainly by Rich HTML editor.
The HCCE’s static string replacement does not do anything for this. Here you want to replace, for instance border=0 with border=”0”. And there could be border=1, or 2, or 3, you never know…

For this, I did not find any turnkey solution. However, I found a guy who solved the similar problem in a different CMS by plugging-in a httpmodule of his own.
http://www.codeplex.com/compfilter4umbraco
He uses this library http://www.codeproject.com/KB/dotnet/apmilhtml.aspx , for basic clean-up of markup.

I was not sure if I can plug-in a httpmodule to MOSS, but since I did not have any other more promising alternative, I gave it a try. And voila, it works!!
For the module, I created it from scratch. I did not need any of logics placed specifically for the CMS. Quicker to build one from scratch than to remove those.
But the library, thanks to the author, I did use it, although I changed it a bit.

The challenge I had was that, at the beginning I had all page go thru this filter. It screwed up things.
So, in the end, I made it so that it cleans up only contents, not part coming from masterpage. That works. This is where the rich editor generating codes appear.

Accessibility : XHTML or HTML

2009-07-18T12:31:00.001-07:00

The first step for an accessible site is conformance to a standard. A colleague of mine who closely follows this subject told me. It can be XHTML or HTML. Does not matter much as far as we conform to one, he added.

Please contest me if I say something stupid, but in a word, a site created using MOSS 2007 Publishing Feature does not conform to none of the two. You can not configure it to. You have to customize it.

It is based on ASP.NET 2.0 which I believe supports XHTML more than HTML.One of the places that you clearly see it is that it inserts lines like below, which violates HTML scheme.

And you can not configure it to stop doing so.

OK. Let us go for XHTML then. But, thank you MS! It violates it too...
You can see it quite easily, just by inserting an image using its Rich HTML Editor. It generates a line like this.

Actually, I have the impression that MOSS Publishing feature is designed(?) to create HTML sites. Its OOTB masterpage files have got HTML doctype declaration.I really do not understand why it comes out like this...

Anyway, after having tried both, I saw going for XHTML should be a realistic solution, and started customizing it.

In a couple of posts after this, I would like to talk about the series of tricks that I employed.

IIS7 Application Request Routing

2009-05-26T02:19:00.000-07:00

While the memory is still fresh…

URL rewrite and/or Reverse Proxy on IIS.
If I am not wrong, it was not possible up to IIS6 i.e. windows 2003, unless you buy a third party tool (which has been our case) or develop something in-house.
Now It has got everything.

I stalled the module (plug-in?) called Application Request Routing.
One small note. When you install it with the version 1.0 installation exe, it installs also the URL Rewrite module, the version 1.0.
It went fine on w2008, the original revision (does the R stand for Rivision?) but not on R2.
I had to install the URL Rewrite module v1.1 separately.

BTW, do we all know what CTP stand for?
The install instruction says that I need to first uninstall the CTP1, if I have. ???
I googled it and found that it is Community Technical Preview, I think.

When I try to install the AAR (Application Request Routing), I had already URL Rewrite v1.1 installed, then found that it does not do reverse proxy.
Since it was not the “CTP”, I did not see it written anywhere, so I went on installing AAR and failed.

Anyway, once you have got it installed, the rest was quite easy. (for me. The goal that I was tasked this time was quite simple. Reverse proxy everything to a given host.)

I created the server farm (of in fact the one host). A pop-up came and said that it would create the routing rule for me. It was just fine. It has turned out to be it.

Customize list form (cont.)

2009-04-24T00:55:00.000-07:00

I wrote on this topic a little bit in the past.
http://murmurofawebmaster.blogspot.com/2008/11/customize-list-form.html

The motivation at that time was, and still is to evaluate, to see what it can, how easily, and what not.
The thing that I picked up (some functionality of the web site to implement using this technique) was the feedback form (I have to have it anyway in one way or another).
The challenges were:
1. It sends an email message. The address is given.
# The current form takes the address from QueryString. With the new SharePoint based website, you find it in the PropertyBag of each site. So idealy, depending on where a link to the feedback is clicked, when it is opened, it should already knows where to send email.
2. An anonymous visitor can give him/her feedback.

For the email, I used the SPD Workflow.
The first prototype takes the address from QueryString, just like our current form. I do not think I can get it from the site’s PropertyBag unless I make the form as an Application Page with code behind.
With some JavaScript, I pick up the email from QueryString and set to a field of the list. You can send a message to it from Workflow.
Nice. Done. (I thought…)

Then, I realized that I can not access the custom insert form of the list anonymously.
After having spent sometime, I now guess that it is probably because the site being in the lockdown mode. http://technet.microsoft.com/en-us/library/cc263468.aspx
OK, then, I can make it as a page under the Pages library.

An error. “The data source control failed to execute the insert command.”
Again spent time on the net to find someone say the following.
“I’m finding out that the dealbreaker with anonymous access is the association with the SPD created workflow. … Design a workflow and associate it with the list and the form will bomb any time that the workflow is initiated.”
http://blogs.devhorizon.com/reza/?p=498

This person ended up doing it with an Application Page with code behind. I guess I would follow the path…

Content Query Web Part

2009-04-08T02:30:00.000-07:00

My goal was to have multilingual summary links.
I tried to implement it through the combination of custom list and content query web part.
Our web site is in multiple languages. Image you have the OOBT summary links placed on pages in different languages, and that you update them one by one. It would be so painful for a person who does not speak the language(s). And it would br very tidious as well.
So my idea was that I have a custom list containing all links, each marked with a language. And using content query part to filter them. For instance if it is on an Arabic page, it picks up only links marked as in Arabic.

First, I defined my multiligual link content type as an inherited type of the built-in link type.
Then I added; language column (built-in), ImageUrl column (custom) of publishing image (comes with the UI that allows my users to browse thru image libraries to select an image) and LinkSortOrder column (custom. Useful also to group links to the same contents in different languages).

Next, I created a custom list of the type.
# I do not think defining the type, and then defining it as the type of the list was required. You could have a list with those columns directly defined. It was a design choice.

Then I prepared a page layout per language. Again a design choice. I may be able to have just one layout to serve all different languages.

Now to customize the presentation.

As you know, it is to write a XSLT code to render the XML returned from CQWP in a way you like.
First, we want to have a look at the XML. I found a good post here http://www.sharepointblogs.com/radi/archive/2009/03/17/content-query-web-part-getting-a-full-dump-of-the-raw-xml.aspx.

However, the challenge was to figure out how to tell the web part to use my XSL rather than the default one.
I have the CQWP placed on a page layout, not a page. I did not find many who do this on the net. At the beginning, I did not think this export/upload was a solution to my case. I gave it a try nevertheless and found out that it is a hidden attribute. “hidden” here I mean, you do not see it by just creating an instance of the OOTB CQWP. So all you have to do is just add it manually.

<PublishingWebControls:ContentByQueryWebPart … MainXslLink="/Style Library/XSL Style Sheets/myContentQueryMain.xsl" … >

In the XML that I finally managed to look into, of course I do not see those columns I added. So I specified the following value to another hidden attribute.

CommonViewFields="URL, URL;ImageUrl, Image"

The URL is a column defined with the built-in Link type.
To determine the types (comes after comma for each column) was a challenge too. It still is, when I do this again for other purposes.
This document http://msdn.microsoft.com/en-us/library/aa981241.aspx is the most explanatory that I found, but still not stisfactorily clear.
For instance, in addition to the URL type, it says now we have another type called Link. I tried both but did not see the difference. They are the same in the XML. I think Microsoft should give us a table where we see the matching between Content Column type and type should be used in the CommonViewFields and other similar properties of CQWP…

Once you have those columns you added appear in the XML, the rest is just usual XSLT coding that many already talk about on the net.

After having gone thru this exercise, I think now that to deal with a custom list such as this one, a better, more beautiful way would be to have the completely own set of columns appear in the XML using yet another hidden attribute called ViewFieldsOverride. There is a nice article http://sharepoint-tweaking.blogspot.com/2008/04/displaying-listname-and-sitename-when.html.

A better still would be that Microsoft, or somebody (though I would rather not to go to a third party solution) come up with a CQWP-like webpart that generates XML according to the custom list and/or content type specified. It may be difficult though, since then what I could have as the default XSL…

Cache-Control: public and Session

2009-04-01T05:55:00.000-07:00

Recently one of our developers reported that session variables are not kept for him and asked me to look into.
After a couple of hours of investigation, I finally came to conclude that this is probably because he set Cache-Control to public in his page.

The below is my conclusion.
When Cache-Control is set tp public with your page, it is undestood that the page is the same for everybody at any time.
When a proxy server sees it, it caches the page, and the cookie, where your session is stored, will not reach the user browser.
Consequently, when the user makes the next request to your server, no cookie will be sent. The server sees it as a new session.

That the cookie does not reach the client is true. I do not see the Set-Cookie header in the response. I would see it if Cache-Control is set to private, the default.
But I was surprised by two things.

One: I found nobody talks specifically about this on the net. I think this is a pitfall that we could fall into easily.
Many says that when the page requires authentication, it has to be private, without explaining why in detail. So you may not see it related.

Two: By the fact that proxy strips cookie when Cache-Control is set to public.
Mmm… Maybe… because it first caches the response and then send it to you. Anyway, it was good to know

default.aspx with custom type and layout under Variations

2009-03-30T01:50:00.000-07:00

Gaaa!! This was difficult and I am not still sure this was the best way.

Objective was (I guess this is for everybody who uses Sharepoint Publishing portal with Variations enabled. For us, this is to support multiple languages) to have a custom content type and a page layout associated with it for default.aspx, when a subsite is created. And the subsite is propagated by Variations as it is.
That is, you create a subsite in the source label. The default.aspx is of your custom type and layout. And you see the same on all target labels.

First, to have default.aspx of a custom type and layout is achieved, by having a custom Site Definition defined and specifying it upon creation of subsites.

If the objective is just this, you may not need to use a custom Site Definition.
(Although it is a bit backdoor-way) I think we could use as well the Site Template technique. The GUI is provided. Thus, it can be done by a series of clicks.
This was not my option because of the Variations.

Suppose you create a subsite in the source label, based on a Site Template with default.aspx being of your custom type and layout.
The propagation of default.aspx fails because the type does not match.
This is because when Variations creates the subsite (propagation) in a target label, it does so based on the Site Template that you selected when you created the source label. It is either Publishing Site with Workflow or Publishing Site. For both, content type of default.aspx is set to Welcome Page.

How to have our custom Site Template selected there? The answer that I found was to come up with a custom Site Definition.
What I did at the beginning was rather simple. I create my definition having the BLANKINTERNET as basis, and just changed the default.aspx provisioning part so my type and layout are used. This what I have in my onet.xml.

<Configuration ID="2">
...
<Modules>
<Module Name="SubWebWelcome" />
</Modules>
</Configuration>
<Modules>
<Module Name="SubWebWelcome" Url="$Resources:cmscore,List_Pages_UrlName;" Path="">
<File Url="default.aspx" Type="GhostableInLibrary" Level="Draft" >
<Property Name="Title" Value="$Resources:cmscore,IPPT_HomeWelcomePage_Title;" />
<Property Name="PublishingPageLayout" Value="~SiteCollection/_catalogs/masterpage/mylayout.aspx, ~SiteCollection/_catalogs/masterpage/mylayout.aspx" />
<Property Name="ContentType" Value="mytype" />
</File>
</Module>

OK. Fine. Looked nice at first. But then I realized that layout is OK but not for the type. The type is Page, not “mytype”.

Actually, up to this far, seems there are many doing the similar. I found blog posts discussing it.
But I found none specifically talking of type. They seemed all happy having their custom layout with default.aspx. Do not tell what happened with the type.
Could be my mistake, oversight somewhere. But I could not figure it out…

After another sometime on the net, I found some people adding custom types to the Pages document library using Feature invoked when a site is created. I gave it a try.
# I later found that the same can be achived using a Feature XML element (http://msdn.microsoft.com/en-us/library/aa543152.aspx), in which case you do not have to do any coding.

Now mytype is added, after the three default types. But still, default.aspx is of type Page, not of my type.

I thought that this may be because the default type of the Pages library is the Page, although it does not make sense because when we create a page thru the UI, it becomes of the type and layout we specify, not of the default type. Actually, when we do so thru the UI, we do not have to add the type first, to begin with.

I looked for a way to change this default (type selected in a document library). But found no such method provided with SPList. It appears that a type is the default just because it comes at the top.

So I decided to see what happens if I deleted the three build-in types, after having added mine.
Bingo!! Now I have the default.aspx created as of my type as well as layout. And since the same definition is used when the site is created in a target label by Variations, there too, the default.aspx is of my type and layout, so the contents are properly propagated.

Zip size limit, 32-bit and 64-bit

2009-03-02T02:46:00.001-08:00

Still FTP migration.

Sometimes one folder is really big. Many files; each rather big. I had problems when unarchiving.
I do not copy it here but from the error message, it seems the upzip can not find the file end.

Found this page. http://www.info-zip.org/FAQ.html#limits
Says the actual size limit for an archive file is 2G on 32-bit and 4G on 64-bit environment.
This explains.

Actually, in my case, the target is 64-bit Windows 2008. We migrate files from a Tru64 UNIX. 4G should be big enough.
However, the info-zip Unzip.exe that I mentioned that I have got in the last post is compiled against 32-bit DLLs, and it reads here (http://www.info-zip.org/board/board.pl?m-1235603338/) that there is no 64-bit Windows version available as of Feb 27/2009.
It says also that I should be able to compile it and can report for any problem. I might want to try that some other time when I am bored but not now.

So I used the –i and x options of zip command to include and exclude some type of file into one but not the other one, to keep one archive less than 2G.
The mission seemed completed but in course of doing this, I have lost the time stamps of some of the files…

Copy data from UNIX to Windows, keeping structure and timestamp

2009-02-24T04:08:00.000-08:00

We are migrating our FTP server from UNIX, Tru64 to a Windows box, Server 2008.
We want to keep the structure and timestamp.

First tried tar.gz, thinking that Server 2008 is smart enough to be able to handle it, without me doing any trick.
No. It does not recognize .gz nor .tar. BTW, I did not give a try to the SUA, subsystem for UNIX-base application.
Since it is already a production server. I wanted to do it as simply as possible. Without installing anything.

I tried the GNU tar, which you do not have to really “install”, just have the executable.
Funnily, the timestamp is not preserved. The date is OK but time changes. We can easily guess that, in which case the time is not really OK either.
Some difference in ways to store datetime info, probably. I did not digg it further.

Then went for ZIP. I should have tried it first. Both platform claim that they support it natively. It was just that I do not use it often.

On UNIX, I zipped the folder structure, just normally without specifying any special option,

% zip -r (resulting zipfile) (folder to archive)
# This is just for me to remember the syntax. The man entry was not so easy to understand...

copied it over to Windows, and simply did the Extract All from the Right-Click menu.
This time, the timestamp of files are OK but not for folders. It appears that folders are created while it unarchives.
Looking into the zip file, without extracting it, shows that the dates are there. So it is just a problem with unarchiving, not with the way the zip file is created.

As far as my research goes, Windows 2008 does not seem to have a command line interface for unzip.
I found one at http://www.info-zip.org/Info-ZIP.html. Seems a dicent project. Downloaded it and finally my folder timestamps are there!!

A small good-to-know with FireBug

2009-02-17T02:06:00.000-08:00

Like me, I think you like it, the FireBug. It is really nice, helpful when especilly you work with CSS.
But have you ever been frustrated when you work on those for hyperlinks i.e. a:link, a:hover etc?
You want to know which of the classes takes the effect when it appears on the browser, but it tells you that only when you click the element.
But with the case with a hyperlink, when you do so, what takes the effect is that of “hover”, not that takes the effect when the page is loaded.

I was very frustrated, helpless, but found out that just by coincident, after you once selected the element, if you refresh the page, it tells you the css class that takes the effect at the page load time.

Business Data Catalog.

2008-12-04T03:54:00.001-08:00

Gaaaaa!!
Lately Sharepoint keeps disappointing me. Here is another one.
We can not make BDC available to anonymous users. http://support.microsoft.com/kb/948729
Since I am using MOSS for an internet facing site, where majority of pages are supposed to be accessible anonymously, this is a big limitation.

And also, I just can not find the exahusted reference listing and explaining all possible values that a Property of an element in the application definision XML can take.

I expected that the BDC Definition Editor in the SDK would do the IntelliSense. But not. It just pops up a blank textbox where you have to type a valid value…

Customize list form

2008-11-14T01:36:00.001-08:00

This is really still just my own memo.

To redirect after submit, add ?source=URL.

The official way (http://office.microsoft.com/en-us/sharepointdesigner/HA101191111033.aspx) of creating a custom list form, uses the DataFormWebPart.
It has a limitation. Attachment does not work. So it has to be disabled.

There seems unsupported workaround exists. http://cid-6d5649bcab6a7f93.spaces.live.com/blog/cns!6D5649BCAB6A7F93!130.entry

If it is just to apply the coporate masterpage, we could go just by replacing that of the default NewForm.aspx with whichever masterpage we want.
Attachment works.

BTW, NewForm.aspx uses ListFormWebPart. We can not customize it, or a very little. For instance, it seems we can not hide fields that we do not need users to fill in.

Input validation. http://rdacollab.blogspot.com/2007/07/custom-sharepoint-edit-forms-with.html

My first experience with Subversion

2008-11-07T08:11:00.001-08:00

On the Subversion server, I did:
- Create a folder for my repository, and
- Did TortoiseSVN “Create repository here …” on the folder.
This should be the equivalent of doing DOS> svnadmin create (path to the folder).

And on my PC,
- Create a folder structure in a temporal location, in the way that I want to organize my projects like below.
Category1\ Project1-1\ trunck
branches
tags
Project1-2\ trunk
branches
tags
Category2\ Project2-1\ trunk
branches
tags
etc.
- Then, import those Categories, specifying the repository’s URL i.e. http://(svn server)/svn/(repository)/.

I wanted to organize projects in categories in one repository, but seems they are igonored when I imported.
What created in the repository are only the projects, in flat structure, no categories…

Then, finally I import my VisualStudio solutions and projects as follows, always on my PC.
For each,
- Import the VS solution, specifying this time URL of the project’s trunk folder i.e. http://(svn server)/svn/(repository)/(project)/trunk.
# my categories were completely ignored…

Here is how to make changes into those projects, and save back into svn.
- First, I need to checkout the VS project from svn, into an empty folder of the name of the project.
# I can not make the change directly into the project from which I did the intial import.
- Then, make whatever modifications I need to do.
- Next, I could verify if nobody else has changed the project in the meantime, with check for modifications, or update.
- Finally, I commit the project.

Variations stopped

2008-10-07T05:52:00.000-07:00

I suddenly realized that the Variations has stopped copying sites and pages from the source to the targets. No recent entries in the log…
Looked into the timer job status and difinition to find that nothing for this particular web application.

I stopped it once intentionally. Unchecked the “Automatic Creation” option. Then I was believing that I had started it again by checking it back.

Found this blog entry http://www.objectsharp.com/cs/blogs/max/archive/2008/02/25/missing-timer-job-definitions-after-sharepoint-move.aspx.
Although what triggerred it is different, the situation is the same. The jobs have disappearred.

I created a “cheating” publishing site collection in the web application in question. The jobs are back. Variations started again.

FlexListViewer

2008-10-01T08:42:00.000-07:00

http://blogs.infosupport.com/porint/archive/2006/08/15/9865.aspx

Have you ever wanted to have a list presented on a page, which resides in a site different from where you have the list?
I believe the list and view are one of the big selling points of Sharepoint. I like them very much. On the contrary, I do not like the Content Query Web Part. Out of question.

Our site is of multi-lingual. We use the Variations technique. Though there are many points of it that I am not happy. This, I believe is one of the biggest shortcomings.
If you say to your users that they have to manage a separate list for each different language, they would laugh at you.

FlexListViewer is my saviour. It allow to have a view displayed in the other site than the one the list belongs to.

Better even, the source code is available. There was a small point of it that does not really fit to our need. We are in the usual, stagin -> internet facing setting. We prepare pages in the staging, and the Content Deployment pushes them to the internet facing site. The out-of-box FlexListViewer, the view URL needs to be a full URL, starting with http://hostname. But in our case, that changes, in the course of deployment. So, do not want to really specify any host.

Thanks to the developper, who made the source available. I modified it so if the specified URL starts with “/”, it assumes the host it is on.

Lock down the Internet facing Sharepoint site

2008-10-01T08:06:00.000-07:00

There is such a command, but this post is not about that. It did not seem to me what I want.

I want to “lock down” the web site so nobody including me can modify it directly. We use the Content Deployment which pushes what we prepare, permission settings as well as contents, from the staging site to the internet facing site. So the modification should happen only at the staging site.

I said we copy the permissions. Our web site is very di-centralized. Every part has own group of people managing it. The management includes “targetting audiences”, restricting access in other words.

The solution I found is the “Policy for Web application”. It allows to say “Deny Write” for everybody.
This is good. Now even the site collection administrators can not modify it.

However then found that even the user used for the Content Deployment can not write into…

Then the solution found is to Extend the Web application, and define Internet zone for the actual service, Denying Write for everybody, while keeping the Default zone still writable.

Need to switch off local Firewall to allow passive mode FTP?

2008-09-05T02:35:00.000-07:00

With IIS7, it is possible to specify a data port range of passive FTP.
This should be a good news for some Firewall administrators.

# I do not know if is was already possible with IIS6. I have never tried to have a serious FTP service on a Windows box.

However, I have the impression that we can not open a range of ports with the Windows local firewall.
Ridiculously, you can only specify a single number…

OK, then what about adding a program to the exception? The deamon process servicing FTP.
You know? With 2008, it appears that many services are running with just one executable, SVCHOST.EXE.
So for example,

C:\>tasklist /SVC
Image Name PID Services
============================================
…
svchost.exe 2880 ftpsvc
…

is the one for FTP I think.

But then if I try to add it to the exception, the system complains. OK, understand, it is almost the same as switching the firewall off…

So, after all this, my conclusion for the moment is that we switch off the local firewall to allow (default for many FTP clients I think) FTP Passive mode.

Follow-up on October 1, 2008:
Found a commnad to issue to “Activate firewall application filter for FTP (aka Stateful FTP) that will dynamically open ports for data connections”.
http://blogs.iis.net/jaroslad/archive/2007/09/29/windows-firewall-setup-for-microsoft-ftp-publishing-service-for-iis-7-0.aspx

Hide Root web from Breadcrumb navigation

2008-09-05T01:21:00.000-07:00

I do not think I am the only one who suffered from this.
If you use the Variations technique to do a multi-lingual site and you want to have a Breadcrumb navigation there.
Then, what you get is: root web > Home > site1 …

But actually you do not want the “root web” AT ALL.
With the Variations, it is just a redirection to one of the Variations root, depending on language setting of the user.
If I have my PC setup language being to French, I get: root web > Accueil > site1 … # “Accueil” means “Home”. It is our top page for French speakers.
So to me, the “root web” appearing there is completely stupid.

I thought it is an everybody’s problem but could not find anybody have come to solve it. I needed to find one myself. Here it is.

I extended the builtin PortalSiteMapProvider, overriding its method GetParentNode() so it returns null for those Variations root node; Home, Accueil etc.

User security information cannot be properly imported without setting UserInfoDateTime option to ImportAll.

2008-07-09T01:23:00.000-07:00

I setup a Content Deployment between my staging and production publishing sites, and got this warning.

For the Content Deployment path, I said All to Security Information. I want to deploy security information (such as ACLs, roles, and membership). This is part of our requirements.

Also, unchecked Deploy user names, following http://technet.microsoft.com/en-us/library/cc263468.aspx.
It reads “… to protect the identities of your authors, disable the Deploy user names setting when you configure the content deployment path.”

Then, it turned out that the above mentioned error, or warning comes from this.

I think I was luckly to find http://msdn.microsoft.com/en-us/library/aa981161.aspx.
The title does not sound relevent, but in the code sample, you see:

// Retain author name during import; change if needed
importSettings.UserInfoDateTime = SPImportUserInfoDateTimeOption.ImportAll;

Why in order to “Retain author name during import”, we need to set UserInfoDateTimeOption to All… Do not ask me…

So I check the Deploy user names, then voila, the warning is gone.

Make a change to an application that uses the Data Access Layer technique

2008-06-30T08:56:00.000-07:00

I think I like this technique in general. I have to admit that the learning curve was high. But once you got used to, I think you would become liking it. Especially if you are a guy like me, who had an ASP doing everying. No “three tier”… but who on the other hand, like OO (object oriented) programming.

As always with a tool like this, a RAD, it is less complicated when you create a brand-new app. But today, I need to add some new fields to the DetailView, to the database table behind.

Of course, you first add the column to the database table.

Then with the TableAdapter representing the table, you should first adapt the main select query. You should not add the column to the DataTable yourself. VS does that for you.

When the change is saved, VS asks you whether you want the update command too adapted. You may find it useful, if you do not do anything special with the update statement, but if you do, ATTENTION, with this auto adaptation, your manual editing of the update statement will be all gone. By the way, it apapts the insert command too, when you say YES.

I had the following difficulty.
As said, I added some new columns to an existing table. One of them was a BOOLEAN field. A bit column in SQL.
I added a CheckBox to the DetailedView for it, and got error saying unable to convert DBNull to BOOLEAN.

OK. Understand. For all existing data it is NULL because I created the column to accept NULL... wait a minute… I could have set the default value…
Anyway, I made it work by setting it to FALSE in the DAL class. First get data in the DataTable calling a GET method of the TableAdapter, then set the field to FALSE if the original value is NULL.

PointFire

2008-06-19T00:30:00.000-07:00

Still, how to do a multi-ligual site.

My boss asked me to evaluate a package called PointFire. Although it turned out not really something we are looking for, it may be interesting for others who have a different set of requirements than ours. But it is not free. I myself find it a little expensive. Please ask directly the vendor about the price.

It does on-the-fly translation. It is a httpmodule, so the translation covers all output, regardless where it is produced; masterpage, layout and contents.

Its user guide reads that for all phrases that Sharepoint uses out-of-the-box like “My Site”, “Site Actions”, it has a built-in dictionary, so Sharepoint own navigation, configuration pages etc are translated.

What do you think happens if you have the sequence of words like “Site Actions” in the middle of your contents? Nothing, no traslation, except that you tag just that two words i.e. <span>Site Actions</span>.

You “register” your own words and phrases that you want the module to translate, into a List called Multilingual Translations created when the module is activated. There I registered the original English phrase “Hello, my name is”, together with its French translation “Bonjour, je m’appele”. And the key is the same. You need to tag a phrase, for it to be translated. I did not see it written anywhere in the userguide though.

Voila, mon rapport bref sur FirePoint. J’espère qu’il vous donne quelque aide…

Re: Sharepoint Variations: Not what I am looking for?

2008-06-18T08:26:00.000-07:00

I posted this question to MSDN forum. And Below is the answer.

“Actually when the page is published it creates a new minor version in all sub variations. It's not overwriting any translated content, because it is a different version of that page.
It's then up to the owner of the content in that target variation to merge any changes into translated content. That's not something you can do by machine, so it's handled by versioning.”

Yes! You are absolutely right. There, you really sometimes find what you are looking for.

Then if the story is really like what I was telling you, just one typo in English, all you have to do at those target sites is to restore the previous version.
Actually, we should not be forced to do this though… There should be a way to correct English typo without affecting others… wait a minute… there may be a way…
I will have a look and report what I found here.

Sharepoint Variations: Not what I am looking for?

2008-06-03T06:23:00.000-07:00

I think I am not going to use it… at least for the moment…. Disappointed very much…

Before, I quoted someone saying as:
“Changes you make to the source site will automatically appear in each target site.”

Ammm… yes, maybe… but in other words, it wipes out what existed previously in the target sites….

That is, OK when the page is first created, but when you then find a typo (one spelling mistake among a million of words), you correct it and re-publish.
The tanslator is emailed, opens the page in one of the target sites to find that… all translations he/she previously did, the million of words are all gone!!!

Reverse proxy from Sharepoint site

2008-05-27T00:49:00.000-07:00

I have been using ISAPI Rewrite 3.0 http://www.helicontech.com/isapi_rewrite/ for URL rewriting at IIS.
I think this is a great tool. Especially for those who use Apache and its Rewrite module for URL rewrite. The sysntax is the same so you can copy your httpd.conf, the part you do some URL rewrite, over to IIS and it works.

It does proxy too. But unfortunately I found it not work with Sharepoint.
This is because Sharepoint uses "Wildcard application maps". Below is extract from its help.
"This table lists ISAPI applications that are executed before a Web file is processed, regardless of the file name extension. These ISAPI applications are called wildcard script maps. Using wildcard script maps is similar to using ISAPI filters, ..."

I removed it and saw what happens. Bingo. My reverse proxy started working. However, now Sharepoint does not behave as before, so I had to put it back.

The ISAPI_Rewrite.dll works with Sharepoint, because it is an isapi filter, so it gets the requests first.ISAPI_RewriteProxy.dll does not, because it is implemented as an isapi extension, rather than a filter.

Request to .rwhlp URL. The aspnet_isapi.dll, which is registered as the wildcard map, takes and returns 404 because it does not exist…

It is really sad. If the proxy too is implemented as an isapi filter, it should work well also with Sharepoint, which I believe is increasing rapidly its share in the market.

Re: Do not see the provisioned application aspx files in “Site Content and Structure”

2008-05-23T02:33:00.000-07:00

The trick was the “type” attribute of <File> element that provisions the aspx.

Previously I had something like:
<Module>
<File Url="PageTemplate.aspx" Name="Default.aspx" Type="Ghostable"></File>
</Module>

With this, the aspx will not appear to the browser interface. I needed to change it to Type="GhostableInLibrary".

So what are the possible values and then what the use of each?

http://msdn.microsoft.com/en-us/library/ms459213.aspx explains the attribute:
“Optional Text. Specifies that the file be cached in memory on the front-end Web server. Possible values include Ghostable and GhostableInLibrary. Both values specify that the file be cached, but GhostableInLibrary specifies that the file be cached as part of a list whose base type is Document Library.
When changes are made, for example, to the home page through the UI, only the differences from the original page definition are stored in the database, while default.aspx is cached in memory along with the schema files. The HTML page that is displayed in the browser is constructed through the combined definition resulting from the original definition cached in memory and from changes stored in the database.”

That means that you have three choices; none (you can go without specifying any value because it is optional), Ghostable or GhostableInLibrary.
And the use of each… Let me stop here for now. I have not found a document explaining it throughly, nor had time to experiment it myself.

Do not see the provisioned application aspx files in “Site Content and Structure”

2008-05-19T05:26:00.000-07:00

Second post of the “ASP.NET application integration into a Sharepoint publishing portal” series.

Application page’s way is fine. I think I have got what I wanted. I have now integrated one of my ASP.NET apps into the Sharepoint portal.
Then moved onto others. There is one intranet application that I would like to integrate into our internal Sharepoint portal.

With it, I implemented its own menu system, controled via a sitemap xml.
It is access control (defined in the web.confg) trimmed. That it, a user would see only menu items that he/she has access.

Once integrated, all this should be already there, provided by the portal. You do not have to implement your own.
Easy!! I only have to locate the provisioned aspx file in the browser interface of Sharepoint and set the permission…

??? I do not see it in the browser interface, on the page titled “Site Content and Structure”...

Feature for an ASP.NET application

2008-05-12T06:10:00.000-07:00

This is first post of the “ASP.NET application integration into a Sharepoint publishing portal” series.

Unlike the way I deployed my CreatePage.aspx-like page (see http://murmurofawebmaster.blogspot.com/2008/05/how-to-have-something-like.html), this time I prepare a Feature for this.

The main difference (to me, at least for the moment) is the masterpage.
With this ASP.NET app, I want to have the masterpage I prepared for my internet facing site. Therefore, it i.e. the aspx page of the app is provisioned through the Feature to the site. If it sits in the LAUOUTS folder, not provisioned into a site, it does not have the masterpage defined for the site.

I did:
1. Created an assembly implementing the code behind class as well as other helper classes. This goes to the Bin folder of the portal.
2. The aspx now inherits a class in the assembly.
3. A Feature to provision, instanciate the aspx to the site, when the Feature is activated.

ASP.NET application integration into a Sharepoint publishing portal

2008-05-12T05:42:00.001-07:00

This is not going to a one-time post to explain the A to Z, but it is really my murmur. I find the solution (I hope I could) as I find problems.

It seems to be a concensus that to this i.e. ASP.NET application integration into Sharepoint, web parts is the way to go.
Below is an extract from a blog post.
1. Convert each Asp.Net page into one or more Asp.Net User Controls.
2. Deploy the controls to the ControlTemplates directory.
3. For each page in the Asp.Net app, create a custom page layout containing its respective User Control(s).
4. Deploy the page layouts to the target site, and create a single page instance for each page layout.

But, frankly, I do not understand why. Why we go for the web parts solution.
We can run an arbitrary aspx page (with code behind class) in a Sharepoint site. This seems simpler to me.

From the next post, I will show you here the steps I took.

Charasters with accent are not displayed correctly

2008-05-09T02:09:00.000-07:00

Our site is in multiple languages. We used to use the iso-8859-1 encoding but needed to move to utf-8 to include languages which use multi-bytes characters like Chinese. But Japanese, my mother tague was not included though…
I want to share with you today the experience, problem and the solution, that we had at the transition.

One day a developper came saying that with his ASP.NET app, characters with accent are not displayed correctly.
I looked into and found that he had the following in the masterpage that he uses by mistake. He did it copying & pasting from an old page.
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />

But at the same time, he had this in the web.config because his site is also in languages with multi-byte characters.
<globalization fileEncoding="utf-8" requestEncoding="utf-8" responseEncoding="utf-8"/>

We correceted the mistake.
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

But the accents still do not come. We spent time looking for hints on the net, and found that the file itself was saved in iso-8859-1.

Visual Studio saved it in iso-8859-1 at the beginning because the materpage it i.e. the aspx file refers to has this.
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />

And it does not change it i.e. does not re-saves the file in the new character encoding, even after the mistake is corrected.
We have to explicitely re-save it in the correct encoding. Here is how to.

You go to “Save as” in the File menu, click the small arrow next to the Save button and choose “Save with Encoding…”.

And choose Unicode on the window opened. # That ISO comes originally tells us that the file is in ISO now.

How to have something like _layout/CreatePage.aspx of our own

2008-05-02T04:51:00.000-07:00

Recently I learnt rather many things thanks to a kind guy I met at MSDN forum, and one of which is this; how to have something like _layout/CreatePage.aspx of your own.

My motivation behind was the following.
I want to have some properties defined with sites. In the very much same sense that we have custom columns, or fields with content types.
I guess many has the same demand.

I thought that an application page could do, and this is going to be very similar to the CreatePage.aspx in the following sense.
- Invoked from the Site Actions menu, and
- Invoked under each site’s context and does what it is supposed to do for the site.
Therefore, the URL has to be http://host/(path to the site)/_layout/theAppPage.aspx.

To do so, you have the manifest.xml as below.
<Solution>
<TemplateFiles>
<TemplateFile Location="LAYOUTS\theAppPage.aspx" />
</TemplateFiles>
</Solution>

When deployed, this will create the aspx file into the C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\LAYOUTS folder, and it becomes available at a URL like above.

Cool!!

How to get error detail with SharePoint

2008-05-02T04:20:00.000-07:00

http://www.keirgordon.com/2007/02/sharepoint-error-detail.html

Let me copy it here too, since the above may one day become unavilable.

In addition to < customerrors mode="Off"/>, which everybody including me could guess, you have to do <SafeMode CallStack="true">.

“please wait while scripts are loaded…”

2008-04-30T05:15:00.000-07:00

I really need to write this… I spent, wasted more than one full day just to sort this small thing…

We implement our corporate page template into a MOSS masterpage.
Copy and paste the html code. Try it first with a blank page layout, not even the page content area. Fine. Looks good and easy.

Add the page content field to the center of layout, to start with.
This time, after creating a content page based on the layout, we type in some text into the page content area at the center.
Do the Preview from the Tools menu. The text I typed DISAPPEAR!! Try to save without previewing. The same. The text is gone… From here, my long debugging exercise started.

It is fine without our page template, the HTML code pasted into the masterpage.
With the template, I noticed that when change the page to edit mode, the status bar of the browser reads “please wait while scripts are loaded…”
Looks something going wrong with javascript codes generated.

A Sharepoint generated page contains tens of thousand lines of javascript. You need a debugger like FireBug to debug this.
We need to either 1 authenticate FireFox against Sharepoint or 2 find a good javascript debugger for IE. Luckily I found the later.

Thanks god! The debugger found the line causing the error. It was in a js file called /_layout/1033/HtmlEditor.js, at its line 5772.

var displayContentElement=document.getElementById(clientId+"_displayContent");
…
var findForm=displayContentElement;
while (findForm.tagName!="FORM" && findForm.tagName!="WINDOW")
{
findForm=findForm.parentElement;
}
findForm.attachEvent("onsubmit",new Function("RTE2_TransferContentsToTextArea('"+clientId+"');"));

At run time, the clientId+"_displayContent” is the id of <div> element for the page content textarea. This code adds an event handler for the form submission, to save the text typed. However, with our page template pasted, the findForm.parentElement does not return. It can not find the <form id=”aspnetForm”> element to add the event handler. Why?

I had the impression that it is probably the <form> element for the search box that we have with the page template, though it does not really make sense because it is properly closed, should not interfere others.
But it was it. Having removed it, my simple page with our corporate page template now works. We think about how to have the search box back later.

To me, the lesson here is that, if you are not lucky and trapped with a problem such as this, it is really difficult to debug it with Sharepoint MOSS.

Feature and Solution, why we need them?

2008-04-29T01:37:00.000-07:00

A Feature is a customization unit that you could activate on a selective unit of SharePoint; a site, a web application, a farm etc. And Solution is packaging framework for Features.

Actually, some built-in “features” of SharePoint are delivered to you as Features.
Built-in masterpages and layouts for Publishing portal for instance are found at C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\FEATURES\PublishingLayouts\MasterPages\.

Likewise, if you create a Solution package containing the masterpage you designed as a Feature, you could deploy it (quite easily indeed) to other instance of SharePoint.

What happens if you do NOT use Feature and Solution? Your customizations? Content types you created with the browser-based interface, layouts you designed with SharePoint Designer etc?

Answer (at least of the moment) is that customization done in that way i.e. thru browser and SPD, goes into database, from which you can not extract and deploy them selectively to other instances of SharePoint. For example, you design your masterpage with SPD, to database. You can not make it available to other instances of SharePoint than the one you created it against, unless you copy and paste the codes manually.

Even worse for contents types. You can not do any copy and paste with them.
For Features, you define them in XML. There are the schema defined.
However, what is painful is that you have to do this all manually. There is no tool yet available to generate them from content database.

Having said that, I do not know if this would be a (major) problem for people like us who maintain own web sites, not consultants or anything who develop solutions for clients.
We have a staging site and production site. Between, we establish Content Deployment job(s). It has been already proven that this “publishes” masterpages and layouts too; everything necessary for content pages to appear correctly, except for assemblies and other file-based customization.
So, what’s wrong if we do all customization to the staging site using browser interface and SPD and they all appear nicely on the production site, without having to worry about deployment etc?

And even, I now wonder if we still need this old "staging site and production site" concept, structure or whatever you may call it.
SharePoint presents a page differently, approved version, draft etc. depending on who you are, where you come from. We might go with just one site.

We may still need the (BROKEN according to many) Content Deployment framework though to distribute contents among servers in a farm. I have not looked into this yet. Will share with you the experience once I had.

Follow-up on July 7 2008:
No, you do not need to setup a Content Deployment to sync contents among a farm.
Actually, it it nothing but obvious. For a farm, you have just one content database (per web application). All web servers in the farm connect to it. You do not need to synchronize anything.

Content Deployment “Timed Out”

2008-04-28T00:44:00.000-07:00

When gone into daylight saving time (DST: this is really IT guys nightmare…) at 2am March 30th 2008, the content deployment stopped working. The result was always “Timed Out”. BTW, to find this, that it may be because of the time change, it took days already….

I found this hotfix http://support.microsoft.com/kb/938663/.
SharePoint’s own timer service, or shceduler, is so smart that it does not notice the time change. Please… Thus, a job scheduled to run at 8am, will not run till 9. When it finally runs, it is already “timed out.”

Since the hotfix in question (http://support.microsoft.com/kb/938535/) is included (http://support.microsoft.com/kb/942390), decided to go for SP1 instead.

One more final remark. Description of the hotfix reads as follows.
“The Windows SharePoint Timer service does not update its internal time when Microsoft Windows makes the transition from standard time to DST or from DST to standard time. Therefore, after you apply this hotfix, you must restart the Windows SharePoint Timer service after each transition from standard time to DST and after each transition from DST to standard time. If you do not restart the Windows SharePoint Timer service, timer jobs that you schedule may be delayed. Or, they may fail.”

Web Statistics; NetIQ WebTrends vs Awstats

2008-04-24T09:20:00.000-07:00

The organisation I work for uses NetIQ WebTrends.
We bought it some years back and have not upgraded it ever since, so I do not know its latest version. The version we use is its eBuisiness edition, version 6.1.

The first thing I noticed was the long time it requires to process log data, especially after we enabled the geographical analysis. Several minutes for just one day.
Our site is a rather busy site, had 1,952,538 hits on Monday for instance, which means you have such number of records in a log file to process. # Actually, it first has to unzip it. This takes some time already.
So it may sound understandable, you may even think it is doing a good job and ask what is the problem.

The problem is that it needs to process the log files, each time you create a new “profile”, to analyze a new thing.
With a profile, you define the “filter” among other things. A filter typically is a URL path, of which you want to have statistics.
For example, this blogger.com server hosts many sites including mine, murmurofawebmaster.blogspot.com. To prepare access statistics of my site, you have to “filter” accesses to this site from the whole gigantic blogger.com server log data.

Imagine that you are towards end of the year, and need to prepare statistics for one particular URL, of the whole year.
For me, it takes even like 3 days. Today, people will not understand that, they will think you are damn.

So we looked at possible alternatives, namely Awstats. I think WebTrends and Awstats are the most famous two in this domain of web log analysis.

First, we listed points that we like and dislike WebTrends.
Good points/Advantage:
G1. Delegation of management
G2. Present geographical distribution of access

Points we dislike:
B(ad)1. A “profile” has to be created in order to see statistics of a subsite independently
B2. Take time for a profile to be ready to be seen
A colleague even said that the way we use it is probably wrong, and there should be a way to make it ready instantly.
B3. The way the profiles are presented/organized. They are just being added chaotically…
B4. Really a black box. An analysis fails without a trace of why it failed.
B5. For something not documented, or that you can not find the explanation, now stuck, because of no valid license. # We decided not to keep its support contract.

Awstats. A perl based freeware.
G1: There is no concept of delegation of management. You administrator need to edit its text config file. -> Disadvantage
G2: Its documentation says possible. But I did not try. The environment I used for this evaluation misses not just one but several required perl libraries. -> Let us say Equal

B1, B2 and B3 -> Equal, or WebTrends is slightly better to me.
I think processing of log data is a time consuming task with all packages in general.
I fed just one day log to Awstats and it took some time (like 10min or so, even more) for it to “digest” even that.

And to have statistics for a subsite, just like with WebTrends, you have to have the specific “profile”, process the log data against the profile and have the result in a separate data store, a database of some sort.

The main difference I found between WebTrends and Awstats with respect to ways to handle series of log data is the following.
With Awstas, it is your responsibility to make sure not to feed it with the same data twice. I think this can be a bit tricky especially when you want to (re-)analyze all past logs. You yourself have to do some programming to achieve this. It is only simple when you feed logs as they are created.
On the other hand, WebTrends remember the full path of log files it has already processed. Therefore, you normally specify a folder and tell it to process everything there. Very simple too when processes all log of this year, today for instance.

B4 and B5 -> Equal
We can not really tell, until we seriously start using it, whether the Awstats is robust, a good set of info for trouble shooting are handy etc.
However, in case of a problem, as with the case with other popular open source packages, I think we could find on the net, people already had that problem and overcome.
With WebTrends, on the contrary, we can not rely on the net. But if you have the support contract, the support guys are there to help you out of whatever problem.

In short, we did not see any significant gain we could have with Awstats, that we do not have with WebTrends. So we decided to stay with WebTrends. We have been using it for some years already (though with some un-satisfaction), and thus we have some level of understanding, know-how, experience etc with it.

Google Analytics

2008-04-24T01:47:00.000-07:00

How many visitors do I have? Looks like this blogger.com does not tell me that.

So I decided to try the Google Analytics. Luckly blogger.com does allows us to insert the required javascript snippets into our blogs.

So far (as anticipated) number of hit I have with this site is really, really small. I need to put more interesting stuff here to draw your attention.

Cynthia Says

2008-04-22T08:55:00.000-07:00

This morning I went to a workshop. It is about to make web site accessible to people with disabilitty.The presentor was Cynthia of http://www.cynthiasays.com/, looks like a very famous person in this area.

I have worked on the topic a little already. My boss asked me to go through the W3C standard and do a summary.It is about stop using <table> to layout the page, or put the labels properly to all input fields of your form etc.

So not many new things came out today’s session. Still, I picked up these interesting from the Q&A session.

PDF is a image, not readable by screen readers.
There are some improvement made by Adobe recently but it is still basically a snapshot image of a document.This is really new to me. I was thinking that it is more like PostsScript than snapshot.

As you may have heard, the Captcha technique is becoming an issue in this context. Our site uses it too.
Today, she presented a couple of possible solutions to it, which include to have audio file next to it. Sounds cool to me.

Massive SEO poisoning

2008-04-18T04:54:00.000-07:00

The story started this blog post. http://ddanchev.blogspot.com/2008/04/unicef-too-iframe-injected-and-seo.html
A guy in the organization I work for, who can be noisy at times, happens to be a subscriber of the feed and bought this to our attentions.
I am the webmaster there, officially titled to be so. So I have to do, or say at least something, when things are brought up this way.

I read the post, hard to understand. Frankly, I do not know if I got the full idea still.
SEO (I did not know the abbreviation) seemingly stands for Search Engine Optimization. In short, in this context, it points to the fact that search engines give higher ranking to pages from “high profile sites”.

Then, IFRAME injection (I did not know that this was getting that popular either) basically is to inject malicious contents using the well-known XSS (Cross Site Scripting) vulnerability.

So I said to the guy that for our site, the XSS was looked into, so we are safe. In reality, you can not be really safe. But you need sometime to be diplomatic, bureaucratic…

The one thing I still do not really get is that, then how to have those injected URLs indexed by google.
According to some posts I found on the net, those malicious guys publish millions of pages tagged with keywords, where they have links to those injected URLs. Google robot comes and is tricked that the injected URLs mentioned at many place for those keywords. It indexes it with a high ranking because it is from a “high profile site”.

FTP : Active vs. Passive

2008-04-17T02:17:00.000-07:00

I keep forgetting this but the occasions I need to understand it keep coming back. Like when I programmed a simple FTP web-based client (I expected I could find a free AJAXed control or something like that, but could not) or help our network guys to do some maintenance. So let me have a brief summary of things good to know/remember so that I do not have to google it yet again…

This is really nice summary I found at http://www.cert.org/tech_tips/ftp_port_attacks.html.
“A client opens a connection to the FTP control port (port 21) of an FTP server. So that the server will be later able to send data back to the client machine, a second (data) connection must be opened between the server and the client.

To make this second connection, the client sends a PORT command to the server machine. This command includes parameters that tell the server which IP address to connect to and which port to open at that address - in most cases this is intended to be a high numbered port on the client machine.

The server then opens that connection, with the source of the connection being port 20 on the server and the destination being the port identified in the PORT command parameters.

The PORT command is usually used only in the "active mode" of FTP, which is the default. It is not usually used in passive (also known as PASV [2]) mode. Note that FTP servers usually implement both modes, and the client specifies which method to use [3].”

Then what is the passive mode? I think this is compact and quick to read but detailed enough, from http://slacksite.com/other/ftp.html#passive.
“In order to resolve the issue of the server initiating the connection to the client a different method for FTP connections was developed. This was known as passive mode, or PASV, after the command used by the client to tell the server it is in passive mode.

In passive mode FTP the client initiates both connections to the server, solving the problem of firewalls filtering the incoming data port connection to the client from the server. When opening an FTP connection, the client opens two random unprivileged ports locally (N > 1023 and N+1). The first port contacts the server on port 21, but instead of then issuing a PORT command and allowing the server to connect back to its data port, the client will issue the PASV command. The result of this is that the server then opens a random unprivileged port (P > 1023) and sends the PORT P command back to the client. The client then initiates the connection from port N+1 to port P on the server to transfer data.”

Develop custom authentication module

2008-04-16T07:48:00.000-07:00

Last week, I enjoyed developping a DNN module specific to our own need.
I do not know how I can best describe it but… the DotNetNuke seems to be a rich framework that we can develop our own customizations on top.

Things, or facilities it exposes for our use inlucde the ease in having properties of the module.
Each module has the end-user interface and configuration interface for the admins to set it up.
You design the configuration interface, listing textbox, checkbox etc for properties of your module. And in the code behind class, you override one method which is called when the admin, after filling those textbox, checkbox etc, clicks the link to save the config change.

I mean, for all these, I do not have to worry about. The framework is there.
What you have to do is just to, taking one of the exsting modues as sample, change the design of the config or enduser intergface or both user controls, and code the event handler method(s).

In addition to that, there are fuctions provided by the framework to save and pick up the value given to a property, into and from database.

I do not know if all this is documented somewhere in a publicly available document. Will let you know if I found it.