Agile practices (Scrum and MSF) in SharePoint -Overview

In SharePoint development along with all other standard development practices we can also use the some agile practices like Scrum and MSF.  Choosing between scrum and MSF might depend on the type of the project and its complexity. As per my knowledge and experience when there is small size project and stake holders ( project owner) want it to be get completed in quick time I will advise to go for scrum practice. This practice is always helpful to understand the project requirement, scope and mutual understanding in very quick time. The delivery is in iteration which helps in bringing together the development team and the stake holders on same page and builds the confidence in stake holders mind that project is moving in right direction. The sprint is the main development unit in this practice and it’s of small duration which helps in controlling the development if it goes in wrong direction. The daily scrum and daily standup helps to resolve any blockers in development by escalating the problem among the team and taking help from any of the members with in or out side team members. This is one of the major advantages in scrum practice.

                   The MSF stands for the Microsoft Solution Framework. This is some set of standard proven development practices. The MSF is mainly categorized in to descriptive and prescriptive components.

In MSF there is no single structure or process. Therefore MSF supports multiple process approaches, so it can be adapted to support any project, regardless of size or complexity. This flexibility means that it can support a wide degree of variation in the implementation of software engineering processes while retaining a set of core principles and mindsets.

The MSF practices can be utilized with the long term and complex type of the projects because it could consists of series of short development cycles and iterations which allow us to continuously learning and refinement in the product.

SharePoint 2010 Remote BLOB Storage (RBS) by Jie Li - SharePoint

I was reading some stuff related to share point remote blob feature. And I got this Awesome short but very important information  from MSDN Jie Li blog. It’s not detail but definitely you can get much more from it.. Enjoy..! :)

Q: What is RBS?

From SQL RBS Team Blog:

RBS(Remote Blob Store) is a set of standardized APIs that allow storage/retrieval of BLOBs outside of your main SQL database where a dedicated BLOB store is desirable for various reasons. This uses a provider model for plugging in any dedicated BLOB store that implements these RBS APIs.

Q: Which version of SQL Server can I use for SharePoint RBS?

SQL Server 2008 and SQL Server 2008 R2 both support RBS. A RBS Library needs to be downloaded and installed on SQL Server to enable the feature. All SQL editions (Express, Standard, Enterprise…) support RBS. Licensing requirement may involve depending on the scenarios.

Q: What is FILESTREAM?

FILESTREAM is a SQL Server 2008 feature to store BLOB content on to file system.

From FILESTREAM Overview:

FILESTREAM integrates the SQL Server Database Engine with an NTFS file system by storing varbinary(max) binary large object (BLOB) data as files on the file system. Transact-SQL statements can insert, update, query, search, and back up FILESTREAM data. Win32 file system interfaces provide streaming access to the data.

FILESTREAM uses the NT system cache for caching file data. This helps reduce any effect that FILESTREAM data might have on Database Engine performance. The SQL Server buffer pool is not used; therefore, this memory is available for query processing.

SQL FILESTREAM feature does not allow to store content on anything other than local storages. (SMB shares cannot be used for store BLOB content)

Q: Is there a comparison of FILESTREAM and RBS?

Please check SQL RBS team blog: SQL Server Remote BLOB Store and FILESTREAM feature comparison.

Q: What is RBS FILESTREAM Provider?

RBS FILESTREAM Provider is a free out-of-the-box provider shipped by Microsoft SQL RBS Team that allows a deployment to use a SQL Database (local or remote) as a dedicated BLOB store. This provider utilizes the FILESTREAM as the BLOB storage mechanism and ties the two technologies together.

Q: Any benefit for using RBS with SharePoint?

By using RBS for SharePoint, customer maybe able to leverage cheaper storage, improve performance, and enable better integration stories with 3rd party technology for their SharePoint databases. But be careful, the benefit is different case by case. You need to investigate your scenarios to see if RBS really fits you.

Q: Any benefit to use RBS FILESTREAM Provider?

Benefit for Windows SharePoint Services 3.0 with Windows Internal Database upgrade:

We no longer support Windows Internal Database (WID) with SharePoint Foundation 2010. However, SQL Server Express has a 4GB DB size limit. The direct upgrade of a content DB > 4gb from WID to SQL Server Express is not supported. To get around of this limit, customer can use RBS FILESTREAM Provider to store all the BLOB content on to file system so the DB size will still remain small.

Benefit for other scenarios:

We are currently recommending to use this technology with document archiving and media streaming scenarios. Tests show that with RBS FILESTREAM Provider large BLOB objects can have better I/O compared with putting everything in the database.

Q: Can RBS FILESTREAM Provider support SMB shares to store the content, for example a NAS device?

No. SQL FILESTREAM feature does not allow to store content on anything other than local storages. Therefore RBS FILESTREAM Provider has the same limitation. 3rd party RBS Providers do not have this limitation if they are not leveraging SQL FILESTREAM feature.

Q: How is the BLOB content stored on the file system with RBS FILESTREAM Provider? Is it encrypted? Does TDE helps with the encryption?

The BLOB objects are stored with GUID names to make sure they can be linked from the RBS tables. The content is not encrypted. TDE does not apply to the content in file stream.

Q: I heard there’s a remote feature in RBS FILESTREAM Provider, is that supported by SharePoint 2010?

This feature allows SQL RBS FILESTREAM Provider to store BLOBs on to a remote SQL Server DB/Instance. SharePoint 2010 will support this remote feature of RBS FILESTREAM Provider.  However the test work will not be finished at the time of SharePoint 2010 RTM. We will announce the supportability of this remote feature later when the test is done.

Q: Is there a licensing requirement for SQL Server when I want to use RBS for SharePoint?

Yes, there is. If the BLOB objects are stored on local SQL Server file system then you can use any SQL Server SKU. For any other scenarios, local SQL Server needs to be Enterprise. The following table shows the license requirement for different scenarios.

Scenario	Local SQL SKU	Remote SQL SKU
FILESTREAM Provider, BLOB stored on local SQL Server file system	Any SQL SKU	N/A
FILESTREAM Provider, BLOB stored on Remote SQL Server file system	SQL Enterprise	SQL Standard or above
3rd Party RBS Provider	SQL Enterprise	N/A

Q: Any 3rd party RBS provider I can use?

Currently StoragePoint (acquired by Metalogix recently) and EMC are providing RBS Providers for different storage options. NetApps, Opentext, Commvault are also working on it.

AvePoint also made their DocAve Extender free – a EBS provider for 2007, and a RBS provider for 2010. (thanks to Chris Musico for pointing out)

STEALTH Software also offers RBS providers. (Thanks Mark)

Q: Is EBS still supported?

EBS(External BLOB Storage API) is introduced by SharePoint 2007. It is still supported in this release (2010). Moving forward, in order to have a more unified storage solution, we will only support RBS in the next release.

Q: How should I backup the content if I’m using RBS?

By using SharePoint Backup and Restore, it doesn’t really matter where the BLOB objects are stored – this is taken care by SQL RBS. So when you use SharePoint Backup on a RBS enabled content database, it will still backup everything. You can restore the backup to a non-RBS enabled database too, the BLOB objects will be inside the database – SQL RBS will decide where to put them, not SharePoint.

SQL Backup now can take care of FILESTREAM BLOB objects.

Q: Can RBS help to remove the 2GB file size limit in SharePoint?

No, using RBS with SharePoint will have the same file size limit.

Q: Can I enable RBS on a content database with content existing? How to migrate those content into the BLOB store? (Thanks to Chris Givens)

Yes, you can enable RBS on a content database and then migrate content into the external storage. There's a "Migrate()" method to handle this process. For example using Windows PowerShell, $cdb is the content db with RBS enabled, the following command will do the job:

$cdb.Migrate()

Q: Should I consider using RBS all the time?

No. RBS may be used when :

• You have huge content dbs for document archiving so you want to reduce storage cost (terabytes of data)
• You have large media files to stream to the audience
• You need to use RBS to integrate 3rd party storage/archiving solutions to SharePoint. (For example EMC Documentum)

For example, if you only have 100GB of data, seperated in several content databases, and most of the content are documents, go for RBS will not benefit your server farm.

Q: Are SQL Mirroring, Clustering and Log Shipping supported when using RBS FILESTREAM Provider?

Log Shipping and Clustering are supported. SQL FILESTREAM cannot be used together with SQL Mirroring.

Service Applications in SharePoint 2010.

Access Services

Use Access Services in Microsoft SharePoint Server 2010 to edit, update, and create linked Microsoft Access 2010 databases that can be viewed and manipulated by using an Internet browser, the Access client, or a linked HTML page.

Business Data Connectivity service

SharePoint Server 2010 and the Microsoft Office 2010 suites include Microsoft Business Connectivity Services, which is a set of services and features that provide a way to connect SharePoint-based solutions to sources of external data and to define external content types based on that external data.

Excel Services Application

Excel Services in Microsoft SharePoint Server 2010 can be used to publish Excel client workbooks on SharePoint Server 2010.

Managed Metadata service

Managed metadata is a hierarchical collection of centrally managed terms that you can define, and then use as attributes for items in SharePoint Server 2010. The Managed Metadata Service supports the use of managed metadata, as well as the sharing of content types across the enterprise.

PerformancePoint Service Application

PerformancePoint Services in Microsoft SharePoint Server 2010 provides flexible, easy-to-use tools for building dashboards, scorecards, and key performance indicators (KPIs).

Search service

With the new capabilities in SharePoint Server 2010, search administrators can configure an optimal search infrastructure that helps end users find information in the enterprise quickly and efficiently.

Secure Store Service

The Secure Store Service is an authorization service that runs on an application server. The Secure Store Service provides a database that is used to store credentials. These credentials usually consist of a user identity and password, but can also contain other fields that you define. For example, SharePoint Server 2010 can use the secure store database to store and retrieve credentials for access to external data sources. The Secure Store Service provides support for storing multiple sets of credentials for multiple back-end systems.

State service

The State Service is a shared service that is used by some Microsoft SharePoint Server 2010 components to store temporary data across related HTTP requests in a SQL Server database. In SharePoint Server 2010, the State Service is required by InfoPath Forms Services (including out of the box and custom workflow forms), the SharePoint Server 2010 Chart Web Part, and certain Microsoft Visio 2010 scenarios that do not use Microsoft Silverlight 3.

Usage and Health Data Collection service

SharePoint Server 2010 includes an integrated health analysis tool called SharePoint Health Analyzer that enables SharePoint Server to automatically check for potential configuration, performance, and usage problems.

User Profile service

The User Profile Service provides our SharePoint farm with all the social networking features that we have come to love in SharePoint 2007, plus more.  It forms the basis of My Site support, User profile pages, Audiences and some of the newer features in SharePoint 2010 social computing such as social tagging.

Visio Graphics Service

The Visio Graphics Service is a service on the SharePoint Server 2010 platform that enables users to share and view Visio diagrams and enables data-connected Microsoft Visio 2010 diagrams to be refreshed and updated from a variety of data sources.

Web Analytics service

The Web Analytics service application is automatically configured as part of the Basic installation of Microsoft SharePoint Server 2010.
Advanced installation of SharePoint Server 2010 requires separate configuration of the Web Analytics service application. You can configure the Web Analytics service application after installation of SharePoint Server 2010 by using one of the following tools:

Word Automation Services

When a conversion job fails, the solution typically provides the string that is displayed in the ConversionItemInfo.ErrorMessage for each failed conversion and directs the user to perform any needed actions on the file. In some cases ConversionItemInfo.ErrorMessage does not contain enough information for an administrator to diagnose and correct a potential system failure

Microsoft SharePoint Foundation Subscription Settings Service

Provides multi-tenant functionality for service applications. Tracks subscription IDs and settings for services that are deployed in partitioned mode. Deployed through Windows PowerShell only.

Application Lifecycle Management in SharePoint 2010

The exact way to develop design ,test and deploy the SOM application in share point.The article is taken from MSDN and may be helpful for sharepoint guys.

The following are the specific steps in the process illustrated in Figure 1 (see corresponding callouts 1 through 10):

1. Someone (for example, a project manager or lead developer) collects initial requirements and turns them into tasks.
2. Developers use Microsoft Visual Studio Team Foundation Server 2010 or other tools to track the development progress and store custom source code.
3. Because source code is stored in a centralized location, you can create automated builds for integration and unit testing purposes. You can also automate testing activities to increase the overall quality of the customizations.
4. After custom solutions have successfully gone through acceptance testing, your development team can continue to the pre-production or quality assurance environment.
5. The pre-production environment should resemble the production environment as much as possible. This often means that the pre-production environment has the same patch level and configurations as the production environment. The purpose of this environment is to ensure that your custom solutions will work in production.
6. Occasionally, copy the production database to the pre-production environment, so that you can imitate the upgrade actions that will be performed in the production environment.
7. After the customizations are verified in the pre-production environment, they are deployed either directly to production or to a production staging environment and then to production.
8. After the customizations are deployed to production, they run against the production database.
9. End users work in the production environment, and give feedback and ideas concerning the different functionalities. Issues and bugs are reported and tracked through established reporting and tracking processes.
10. Feedback, bugs, and other issues in the production environment are turned into requirements, which are prioritized and turned into developer tasks. Figure 2 shows how multiple developer teams can work with and process bug reports and change requests that are received from end users of the production environment. The model in Figure 2 also shows how development teams might coordinate their solution packages. For example, the framework team and the functionality development team might follow separate versioning models that must be coordinated as they track bugs and changes.

Issues occuring while synching the MPP file with sharepoint tasklist

We can synch up the Microsoft project plan with SP task list and once it’s in share point we can implement or build our own controls tools on the data which is available in task list.

To get more idea how to synch the mpp file with task/project list please visit here

The synch up has some limitations that must be known before doing any synch up. Some of them are as follows.

SharePoint can only synchronize resources that exist within its environment

When same item has been modified both in SharePoint and Microsoft Project. In this example the %complete has been modified in both sides with different values. The synchronization process pops up the conflict resolution windows to help you resolving this issue.

Only works with SharePoint Foundation or SharePoint Server 2010

Only works with Project Professional
only works when Project Professional is not connected to Project Server
Synchronized tasks become Manually Scheduled tasks (cannot keep Auto Scheduled

Few days back I was working with Microsoft Project professional and share-point task list. We have to synch up the MPP file with share-point task list. It got synched for first time but next time when we downloaded it locally and tried to synch it started giving some unexpected error messages.

Some of them are like..

Unable to Synch the file because the two or more summary task name have the same name. 

We checked the file for duplicate summary name and we tried it once more by removing the duplicate names then it got synched up but this is not happening on regular basis. After removing the duplicate summary name  some times it synch up some times not and sometimes it synch also the duplicate summary name.

Another error I got was... 

Unknown error has occurred unable to synch up with share point list Check your share point site is running.

This is really something annoying to me. Because the site is ON/Live, and I was able to browse and down load from it properly.  :)

To resolve all above issues I did some goggling and found some solutions around it. But none of them are 100% solving my problem.

Some people says

Links can only be Finish to Start;

External dependencies are prohibited;

Custom fields are problematic i.e. sometimes will sync, then subsequent runs will fail.
Is any body  SME in this...?

Sharepoint Workflow status constants

• Status: Value
• Not Started: 0
• Failed on Start: 1
• In Progress: 2
• Error Occurred: 3
• Canceled: 4
• Completed: 5
• Failed on Start(Retrying): 6
• Error Occurred (Retrying): 7
• Canceled: 15
• Approved: 16
• Rejected: 17

Silver light-SharePoint development tips

Silverlight provides very rich UI and it can used very well to communicate with SP2010. But there are a lot of hidden issues how to develop, debug and test SL. In this post I will share the challenges I experienced with Silverlight in SharePoint 2010 and solutions to help others to cut the corners.

1) Development

With SharePoint 2010 you can use Silverlight 3.5 or Silverlight 4.0. We had no dependency on .NET Framework version so picked the decent 4.0 version. You need to install Silverlight 4.0 Tools, SDL, WCF

When you create a new Silverlight project you will be asked if you want to host it in the external Web Application. In all articles I read they recommend not to host in external webApp and build and deploy directly to SharePoint via Features . It’s up to you if you choose that way, but for me deploying to SharePoint and testing form there is very annoying and time consuming process (it builds WSP, deploys WSP, resets IIS, SharePoint page recompiles), so I prefer to develop and deploy to the simple aspx page and then deploy to SharePoint when necessary. The development is much faster in this case.

2) Web Services

To communicate with external systems you need to make calls ether directly from Silverlight or using Web Services. Take into account that Silverlight is the client technology and all calls will be made from the client's browser (similar to making call to external system via javascript – JSON and REST models is what you need). It’s ok to make calls to SharePoint Object Model (Lists, Sites and etc) directly from Silverlight, but working with Database will be quire expensive in terms of loading your server with requests. In this case you need to consider creating the Web Service and call database from there.

Silverlight has limited support of SOAP services, so, consider using WCF Services, due to better support of serialized types.

3) Debugging

SharePoint Silverlight debugging in Visual Studio is not supported by default, due to Script debugging settings turned on. It means that you can’t set breakpoints and step into your code in VS if you are not deploying Silverlight controls via feature. Such functionality is supported when you host your SL project in external Web Project (another reason not to host in SharePoint during development)

To provide the SharePoint SL debugging capabilities you need to package your .XAP file in SharePoint Project (in VS) and deploy as a Feature. Such approach allows to enable “Siverlight Debugging” instead of client one, and step into your .cs code from Visual Studio 2010. This options is set from the VS IDE. Navigate to the SharePoint project properties –> SharePoint Tab –> click “Enable Silverlight Debugging”. That checkbox will activate the Remote Debugger and you will be able to debug SL hosted in SharePoint using Visual Studio 2010 (but you need to configure the Remote Debugger first, opening the firewall ports. On the 64bit windows you need to configure the x86 version of msvsmon.exe because the Visual Studio is x86)

I found really strange behaviour that debugging doesn’t work if you don’t have another instance of the the SL page opened in browser. Seems there is a bug resolving assemblies, because when you start debugging your solution might be re-builded , IIS restarts, thus remote debugger can’t attach to the right assemblies. To fix it just open another tab in browser, navigate to page with SL control you are debugging and only after that start debugging which will open the second instance of that page.

4) Deployment

You can deploy your Silverlight controls (XAP files) either externally Take into account that if you host Web Service outside Silverlight you need to supply ClientAccessPolicy.xml and crossdomain.xml for the cross-domain calls. Create and put these files into the root folder of SharePoint (\inetpub\wwwroot\wss\VirtualDirectories\) or into the root of the external hosing(depending the way you host SL and services)

To deploy the XAP files to SharePoint they recommend to use the following locations: SharePoint Library, “_catalogs/masterpage/”, “_layouts” or custom locations. I found that the standard SharePoint 2010 controls are stored into the “_layouts/clientbin/” folder. This folder is designed to be a standard place for hosting assemblies that are used in Silverlight. Take into account that all files deployed to SharePoint are ghostable (even you don’t specify the type), and you won’t see them on physical drive - use the SharePoint Designer to check the the file presence in the folder you deployed it.