SharePoint .NET client object model against load balanced servers

This, I found nowhere on the net the solution, and so nearly given up.

For the SharePoint client object model, the sample code are available everywhere on the net e.g. https://docs.microsoft.com/en-us/sharepoint/dev/sp-add-ins/complete-basic-operations-using-sharepoint-client-library-code. None worked for me. The error was “(401) Unauthorized”. The problem was that as you know, even though the servers, a SharePoint farm in this case, are loaded balanced with whatever actual software and/or hardware in use to do so, to perform operations only the authorized users can perform, you need to interact with a single server. There seem, however, no documentation available (I was not able to find at least) explaining how to achieve this.

And so this is how I made it work. Bear in mind that I am not from Microsoft or anything, not an authority, so cannot guarantee you that this is the way to go.

and then

What I expect with this was to accept the load balancer issuing sticky cookie and send it back with the subsequent requests. And it seems do the trick.

Hope this helps you.

Hit highlighted summary

Haven’t you ever wondered why for some results, the summary does not contain any highlighted word?

Although the found document file does contain the searched word.

After having spent sometime, I think I found the explanation.

Basically, it is explained at http://goo.gl/RiSFBx.

In short, SharePoint keeps only 10,000 characters for each indexed file for the summary, and highlights the searched word only if it appears in the saved text.

WSS_KeepSessionAuthenticated

As becoming the de-fact, when asking users to authenticate, we change i.e. redirect to SSL. But then, if he/she opens another browser window and comes to us on http, to view two pages from our site simultaneously, on the second windows, he/she goes back anonymous. He/She has authenticated on the SSL i.e. https session, but not on http.

We first implemented this mechanism on SharePoint 2007. We may have wanted to redirect the second session to SSL so that the user finds himself logged-in on both windows. But due to some technical constraint not really of SharePoint, it was not that straightforward.

Anyway, this is how we started.

Then, when upgraded to 2010, we saw that the user is asked for username/password on the second session, even though the page that he is trying to open is anonymously accessible. After some research, we found out that this seems due to the WSS_KeepSessionAuthenticated cookie, that is sent. SharePoint forces the user to authenticate when it finds the cookie in the request.

To overcome, we created and hooked up a HttpModule to clear it at the beginning of the request made on http. It is an acceptable workaround for us, because we do not ask users to send their passwords as clear text.

Today, I found out that it is no longer needed with SharePoint 2013. The problem was that with SP2010, the cookie was set as non-secure. It is set when the user authenticates, and sent back to the server whether the session is secure or not. While with SP2013, it is set as secure when authenticated on https, and is sent back only when the communication is secured i.e. on SSL.

SharePoint2013 migration resetting master page references

If I am not wrong, it does, isn’t it?

Actually, it is not new. It did so when migrating to 2010 also. It changed master page reference of all sites to v4.master. I wrote a code to reset them back to our custom master. Needless to say that, before doing that, the custom master needed to be adapted. I needed the code to do this, because, even with sites where you see the master page reference is inherited from its parent, the effective master page became to v4.master.

I did not think it a big flaw at that time, but now think it is. What if for a legitimate reason, an organization employs several masters for their public facing web site, say for instance, employing different brandings for different products, champagnes etc.? Once upgraded, you do not know any more which site was on which master.

If SharePoint does not allow such an arrangement i.e. employing different master page for different sites, then OK. We would not complain. But it does allow, even encourages you, and says NO you cannot, all the sudden, when a new version comes out.

I did some googling, to find out that not so many seem concerned. Is this the sign that it has never gained enough popularity in the public facing web sites market? Where obviously branding is one of the most important aspects.

[MOSS 2007 public facing site] access to MS Office documents in Document Library

I am less and less certain if this is a right product…
This is THE most incredible and discouraging experience that I ever had with this product, which I have been working on to maintain our public internet site.

First of all, MANY MANY MANY thanks to the author of this blog post.
http://www.theblackknightsings.com/RemoveLoginBoxWhenAnonymousUsersDownloadOfficeDocumentFromSharePointSite.aspx
# BTW, the design is very funny. I thought it was a genuine error screen at the beginning.

The explanation is very clear, and so is the code. So I went straight plugging it in to our production site.
# I have at least compiled the DLL myself, and did a bit of test though.

We know that this product i.e. SharePoint is now widely used, especially with intranets, sites for collaboration and so on, where you know the users, they are all authenticated. However, for public facing internet sites such as ours where majority of users are anonymous, there are points overlooked, including ones as obvious as this one. And we users by ourselves need to apply workarounds… Hope with the coming 2010, it comes as a WCM product fully ready for public sites.

"The site with the id {GUID} could not be found" and BLOBCache

My own memo. Nothing I myself found.
Anyway it was really hard to figure out. But I was lucky that it happened in our testing environment. So I did not have to start crying yet.

The problem was the following.
I was getting “The site with the id {GUID} could not be found” error to get some css files from Style Library. But only on the extended web application, and futhermore, only when a user logs in to the site. # Our site accepts anonymous visitors too.

I re- extended the web app, re-restored the data from our production site. Nothing helped. # I did not have the problem on the production site.

As always, I went to the net for some help, a hint at least. But this time I did not think I would find anything helpful. I was believing that the content database might have been corrupted somehow, or something…

However, there are people who had the exact same problem, and have overcome it.
http://blogs.msdn.com/joshuag/archive/2008/05/22/filenotfoundexception-the-site-with-the-id-guid-could-not-be-found.aspx

This was a big help. I did not even think about the cache.
Unfortunately, however, the solution did not work for me.

But I know now all I need to know is how to clear the cache. And found this http://blogs.pointbridge.com/Blogs/monnette_jeff/Pages/Post.aspx?_ID=15.

More (seemingly) in depth explanation can be found at http://sharepointinterface.com/2009/06/18/we-drift-deeper-into-the-sound-as-the-flush-comes/.
I did not have anough patience to read it throughly though…

In short, the UI does not work for a farm like ours, and the stsadm solution (http://msdn.microsoft.com/en-us/library/aa622758.aspx) stinks.
We have only two front-ends. So I went for the manual flush.

Accessibility : Accessibility Kit for Sharepoint

There is this tool (? add-on?) exists. But after all it did not help much.

You find a couple of things in the package but we were only interested in what they call HCCE (HiSoftware Compliant Code Engine) and WebZonePart controladapter.

The HCCE is nothing more than a custom base class for Page Layout, and what it does is simple string replacements.
For instance, MOSS Publishing feature generates <SCRIPT> in its output HTML, which violates XHTML. It has to be <script type=”text/javascript”>.
It does this, the replacement in the output. But only if you specify it, exactly as above, one by one, in its config file.
So it is not like, it does the work if you say you want to “comply” to XHTML, or HTML. “Compliant Code Engine”… Funny..

And the other cr*p, WebZonePart controladapter, it is a controladapter. So as you know, it replaces with table with div.
By default, MOSS generates lines like below for each webpart on the page. Of course it all violates standard.
<td id="MSOZoneCell_WebPartWPQ3" orientation="Vertical" name="MSOZoneCell" relatedWebPart="WebPartWPQ3"…

The adapter replaces those with divs. Sounds nice. But my problem was that it does not produces title and border of the part even if you configure you want them.
Luckly, the package contains the source (not for HCCE. HCCE comes only as binary (of debug build…)). So I managed to have them back.


What still remains not OK are those XHTML not compatible codes generated mainly by Rich HTML editor.
The HCCE’s static string replacement does not do anything for this. Here you want to replace, for instance border=0 with border=”0”. And there could be border=1, or 2, or 3, you never know…

For this, I did not find any turnkey solution. However, I found a guy who solved the similar problem in a different CMS by plugging-in a httpmodule of his own.
http://www.codeplex.com/compfilter4umbraco
He uses this library http://www.codeproject.com/KB/dotnet/apmilhtml.aspx , for basic clean-up of markup.

I was not sure if I can plug-in a httpmodule to MOSS, but since I did not have any other more promising alternative, I gave it a try. And voila, it works!!
For the module, I created it from scratch. I did not need any of logics placed specifically for the CMS. Quicker to build one from scratch than to remove those.
But the library, thanks to the author, I did use it, although I changed it a bit.

The challenge I had was that, at the beginning I had all page go thru this filter. It screwed up things.
So, in the end, I made it so that it cleans up only contents, not part coming from masterpage. That works. This is where the rich editor generating codes appear.

Accessibility : XHTML or HTML

The first step for an accessible site is conformance to a standard. A colleague of mine who closely follows this subject told me. It can be XHTML or HTML. Does not matter much as far as we conform to one, he added.

Please contest me if I say something stupid, but in a word, a site created using MOSS 2007 Publishing Feature does not conform to none of the two. You can not configure it to. You have to customize it.

It is based on ASP.NET 2.0 which I believe supports XHTML more than HTML.One of the places that you clearly see it is that it inserts lines like below, which violates HTML scheme.

<input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="..." />

And you can not configure it to stop doing so.

OK. Let us go for XHTML then. But, thank you MS! It violates it too...
You can see it quite easily, just by inserting an image using its Rich HTML Editor. It generates a line like this.

<img border=0 ... >

Actually, I have the impression that MOSS Publishing feature is designed(?) to create HTML sites. Its OOTB masterpage files have got HTML doctype declaration.I really do not understand why it comes out like this...

Anyway, after having tried both, I saw going for XHTML should be a realistic solution, and started customizing it.

In a couple of posts after this, I would like to talk about the series of tricks that I employed.

Customize list form (cont.)

I wrote on this topic a little bit in the past.
http://murmurofawebmaster.blogspot.com/2008/11/customize-list-form.html

The motivation at that time was, and still is to evaluate, to see what it can, how easily, and what not.
The thing that I picked up (some functionality of the web site to implement using this technique) was the feedback form (I have to have it anyway in one way or another).
The challenges were:
1. It sends an email message. The address is given.
# The current form takes the address from QueryString. With the new SharePoint based website, you find it in the PropertyBag of each site. So idealy, depending on where a link to the feedback is clicked, when it is opened, it should already knows where to send email.
2. An anonymous visitor can give him/her feedback.

For the email, I used the SPD Workflow.
The first prototype takes the address from QueryString, just like our current form. I do not think I can get it from the site’s PropertyBag unless I make the form as an Application Page with code behind.
With some JavaScript, I pick up the email from QueryString and set to a field of the list. You can send a message to it from Workflow.
Nice. Done. (I thought…)

Then, I realized that I can not access the custom insert form of the list anonymously.
After having spent sometime, I now guess that it is probably because the site being in the lockdown mode. http://technet.microsoft.com/en-us/library/cc263468.aspx
OK, then, I can make it as a page under the Pages library.

An error. “The data source control failed to execute the insert command.”
Again spent time on the net to find someone say the following.
“I’m finding out that the dealbreaker with anonymous access is the association with the SPD created workflow. … Design a workflow and associate it with the list and the form will bomb any time that the workflow is initiated.”
http://blogs.devhorizon.com/reza/?p=498

This person ended up doing it with an Application Page with code behind. I guess I would follow the path…

default.aspx with custom type and layout under Variations

Gaaa!! This was difficult and I am not still sure this was the best way.

Objective was (I guess this is for everybody who uses Sharepoint Publishing portal with Variations enabled. For us, this is to support multiple languages) to have a custom content type and a page layout associated with it for default.aspx, when a subsite is created. And the subsite is propagated by Variations as it is.
That is, you create a subsite in the source label. The default.aspx is of your custom type and layout. And you see the same on all target labels.

First, to have default.aspx of a custom type and layout is achieved, by having a custom Site Definition defined and specifying it upon creation of subsites.

If the objective is just this, you may not need to use a custom Site Definition.
(Although it is a bit backdoor-way) I think we could use as well the Site Template technique. The GUI is provided. Thus, it can be done by a series of clicks.
This was not my option because of the Variations.

Suppose you create a subsite in the source label, based on a Site Template with default.aspx being of your custom type and layout.
The propagation of default.aspx fails because the type does not match.
This is because when Variations creates the subsite (propagation) in a target label, it does so based on the Site Template that you selected when you created the source label. It is either Publishing Site with Workflow or Publishing Site. For both, content type of default.aspx is set to Welcome Page.

How to have our custom Site Template selected there? The answer that I found was to come up with a custom Site Definition.
What I did at the beginning was rather simple. I create my definition having the BLANKINTERNET as basis, and just changed the default.aspx provisioning part so my type and layout are used. This what I have in my onet.xml.

<Configuration ID="2">
...
<Modules>
<Module Name="SubWebWelcome" />
</Modules>
</Configuration>
<Modules>
<Module Name="SubWebWelcome" Url="$Resources:cmscore,List_Pages_UrlName;" Path="">
<File Url="default.aspx" Type="GhostableInLibrary" Level="Draft" >
<Property Name="Title" Value="$Resources:cmscore,IPPT_HomeWelcomePage_Title;" />
<Property Name="PublishingPageLayout" Value="~SiteCollection/_catalogs/masterpage/mylayout.aspx, ~SiteCollection/_catalogs/masterpage/mylayout.aspx" />
<Property Name="ContentType" Value="mytype" />
</File>
</Module>

OK. Fine. Looked nice at first. But then I realized that layout is OK but not for the type. The type is Page, not “mytype”.

Actually, up to this far, seems there are many doing the similar. I found blog posts discussing it.
But I found none specifically talking of type. They seemed all happy having their custom layout with default.aspx. Do not tell what happened with the type.
Could be my mistake, oversight somewhere. But I could not figure it out…

After another sometime on the net, I found some people adding custom types to the Pages document library using Feature invoked when a site is created. I gave it a try.
# I later found that the same can be achived using a Feature XML element (http://msdn.microsoft.com/en-us/library/aa543152.aspx), in which case you do not have to do any coding.

Now mytype is added, after the three default types. But still, default.aspx is of type Page, not of my type.

I thought that this may be because the default type of the Pages library is the Page, although it does not make sense because when we create a page thru the UI, it becomes of the type and layout we specify, not of the default type. Actually, when we do so thru the UI, we do not have to add the type first, to begin with.

I looked for a way to change this default (type selected in a document library). But found no such method provided with SPList. It appears that a type is the default just because it comes at the top.

So I decided to see what happens if I deleted the three build-in types, after having added mine.
Bingo!! Now I have the default.aspx created as of my type as well as layout. And since the same definition is used when the site is created in a target label by Variations, there too, the default.aspx is of my type and layout, so the contents are properly propagated.

Business Data Catalog.

Gaaaaa!!
Lately Sharepoint keeps disappointing me. Here is another one.
We can not make BDC available to anonymous users. http://support.microsoft.com/kb/948729
Since I am using MOSS for an internet facing site, where majority of pages are supposed to be accessible anonymously, this is a big limitation.

And also, I just can not find the exahusted reference listing and explaining all possible values that a Property of an element in the application definision XML can take.

I expected that the BDC Definition Editor in the SDK would do the IntelliSense. But not. It just pops up a blank textbox where you have to type a valid value…

Customize list form

This is really still just my own memo.

To redirect after submit, add ?source=URL.

The official way (http://office.microsoft.com/en-us/sharepointdesigner/HA101191111033.aspx) of creating a custom list form, uses the DataFormWebPart.
It has a limitation. Attachment does not work. So it has to be disabled.

There seems unsupported workaround exists. http://cid-6d5649bcab6a7f93.spaces.live.com/blog/cns!6D5649BCAB6A7F93!130.entry


If it is just to apply the coporate masterpage, we could go just by replacing that of the default NewForm.aspx with whichever masterpage we want.
Attachment works.

BTW, NewForm.aspx uses ListFormWebPart. We can not customize it, or a very little. For instance, it seems we can not hide fields that we do not need users to fill in.

Input validation. http://rdacollab.blogspot.com/2007/07/custom-sharepoint-edit-forms-with.html

Variations stopped

I suddenly realized that the Variations has stopped copying sites and pages from the source to the targets. No recent entries in the log…
Looked into the timer job status and difinition to find that nothing for this particular web application.

I stopped it once intentionally. Unchecked the “Automatic Creation” option. Then I was believing that I had started it again by checking it back.

Found this blog entry http://www.objectsharp.com/cs/blogs/max/archive/2008/02/25/missing-timer-job-definitions-after-sharepoint-move.aspx.
Although what triggerred it is different, the situation is the same. The jobs have disappearred.

I created a “cheating” publishing site collection in the web application in question. The jobs are back. Variations started again.

FlexListViewer

http://blogs.infosupport.com/porint/archive/2006/08/15/9865.aspx

Have you ever wanted to have a list presented on a page, which resides in a site different from where you have the list?
I believe the list and view are one of the big selling points of Sharepoint. I like them very much. On the contrary, I do not like the Content Query Web Part. Out of question.

Our site is of multi-lingual. We use the Variations technique. Though there are many points of it that I am not happy. This, I believe is one of the biggest shortcomings.
If you say to your users that they have to manage a separate list for each different language, they would laugh at you.

FlexListViewer is my saviour. It allow to have a view displayed in the other site than the one the list belongs to.

Better even, the source code is available. There was a small point of it that does not really fit to our need. We are in the usual, stagin -> internet facing setting. We prepare pages in the staging, and the Content Deployment pushes them to the internet facing site. The out-of-box FlexListViewer, the view URL needs to be a full URL, starting with http://hostname. But in our case, that changes, in the course of deployment. So, do not want to really specify any host.

Thanks to the developper, who made the source available. I modified it so if the specified URL starts with “/”, it assumes the host it is on.

Lock down the Internet facing Sharepoint site

There is such a command, but this post is not about that. It did not seem to me what I want.

I want to “lock down” the web site so nobody including me can modify it directly. We use the Content Deployment which pushes what we prepare, permission settings as well as contents, from the staging site to the internet facing site. So the modification should happen only at the staging site.

I said we copy the permissions. Our web site is very di-centralized. Every part has own group of people managing it. The management includes “targetting audiences”, restricting access in other words.

The solution I found is the “Policy for Web application”. It allows to say “Deny Write” for everybody.
This is good. Now even the site collection administrators can not modify it.

However then found that even the user used for the Content Deployment can not write into…

Then the solution found is to Extend the Web application, and define Internet zone for the actual service, Denying Write for everybody, while keeping the Default zone still writable.

Hide Root web from Breadcrumb navigation

I do not think I am the only one who suffered from this.
If you use the Variations technique to do a multi-lingual site and you want to have a Breadcrumb navigation there.
Then, what you get is: root web > Home > site1 …

But actually you do not want the “root web” AT ALL.
With the Variations, it is just a redirection to one of the Variations root, depending on language setting of the user.
If I have my PC setup language being to French, I get: root web > Accueil > site1 … # “Accueil” means “Home”. It is our top page for French speakers.
So to me, the “root web” appearing there is completely stupid.

I thought it is an everybody’s problem but could not find anybody have come to solve it. I needed to find one myself. Here it is.

I extended the builtin PortalSiteMapProvider, overriding its method GetParentNode() so it returns null for those Variations root node; Home, Accueil etc.