Why did Microsoft create the SharePoint Framework (SPFx)?

This is one of a multi-part series of posts answering the most common questions on the SharePoint Framework. This series, SharePoint Framework Five “W"s & One “H” answered , gives you the best high-level picture of the SharePoint Framework. I ( Andrew Connell ) wrote this series in late 2020 to help people new to the SharePoint Framework get the answers to the most common and basic questions about the SharePoint Framework.

When you try to learn or understand something new, I’ve always found the best approach is to first get a grasp on answering the following question: why. The answer to this question gives you such valuable insight into the motivations, reasons and goals for the thing you want to learn.

In the case of the SharePoint Framework (SPFx), I find most of the questions I get from people new to SPFx would resolve themselves if they had taken some time to get a little perspective on history and context. Answering the question Why did Microsoft create the SharePoint Framework requires you to have some history under your belt. That’s what I’ll do in this article – give you a little SharePoint history lesson.

This is going to be a rather lengthy article because it takes time to understand where we came from. Before I do that, let me answer my own question:

Why do I need a SharePoint history lesson to understand why Microsoft created the SharePoint Framework?

The reason is simple: as SharePoint evolved over time into the product we have today, one that is primarily a hosted implementation in SharePoint Online, Microsoft iterated through multiple development models until they found one that worked for both developers & Microsoft alike. They came to this conclusion by addressing the challenges and requires they were faced and by observing what customers were doing through their implicit actions.

So… let’s get started!

2003 – 2009: Full trust solutions - SharePoint Portal Server 2003 (WSS v2) & Office SharePoint Server 2007 (WSS v3)

While SharePoint’s roots go back before 2003 to the days when we called it Tahoe, I want to start in 2003.

In SharePoint Portal Server 2003 & Windows SharePoint Services v2 (SPS 2003 & WSS v2), Microsoft shipped a product that was designed as a departmental portal and collaboration space. Developers weren’t given any sort of a development model, but if you knew where to look, you could create custom solutions.

In [Microsoft] Office SharePoint Server 2007 & Windows SharePoint Services v3 (MOSS 2007 & WSS v3), Microsoft introduced two concepts that were a huge step forward for custom solutions: Features & Solutions. Features were effectively plugins of reusable functionality you could add to SharePoint and solutions were the packaging & deployment vehicle to get this custom code added to SharePoint & the server. Both of these things are still with us today in modern date SharePoint. Features and solutions are still used to deploy SPFx solutions in SharePoint.

When developers built custom solutions during this time, in either SPS 2003 or MOSS 2007, they could do anything they wanted to do with the API. This allowed us to build some very powerful and robust solutions, but there were also some drawbacks.

Full trust solutions

These customizations executed within the same process as the SharePoint process. This means they ran under full trust, which means they had full access to the SharePoint API enabling some powerful customizations. But the downside was that if the custom code ever failed and crashed, it would bring down the entire process, including the hosting SharePoint process. Yup, that means if your custom stock ticker web part threw a divide by zero exception, your entire corporate intranet was also crashing.

At the time we called these solutions, but looking back, these were later named full trust solutions for reasons that will be clear in the next section.

SharePoint was expensive & hard to run

Around this time, Microsoft started looking more closely at the business of SharePoint. At this time, they saw how SharePoint was mostly only available to large organizations. It was expensive to license, expensive to host, required a lot of domain knowledge in that administrators had to know not only how to manage SharePoint, but also Windows & SQL Server to name just a few other products. This put SharePoint out of many medium sized companies and most small businesses.

The workings of a hosted SharePoint service started to grow. First it happened in the ecosystem, and then Microsoft wanted to get into the game. To do this at scale, you really needed to have the ability to put multiple customers on the same SharePoint installation. One customer per server just wasn’t going to scale. We call this architecture single-tenant in that one customer (aka: tenant)

But the idea allowing customers to deploy their customizations to a server where one broken web part could bring down the process not just for their intranet, but also those sites of other customers on the same server wasn’t going to work.

So… they needed a better option.

2010 – 2012: Partial trust solutions - SharePoint Server 2010 & SharePoint Online

In SharePoint Server 2010, Microsoft introduced a few new things to make SharePoint more multi-tenant friendly. They introduced the concept of service applications which allowed administrators to create a process, like a search indexer, that was separate from a site and could be shared by multiple sites.

Partial trust solutions

But for developers, they introduced the sandbox. The sandbox was a separate process from the core SharePoint process that would run sandboxed solutions. When a web part in a sandboxed solution was requested on a page, the core SharePoint process would call the sandbox process to run the web part and get the results to display in the SharePoint page.

This meant that if your custom code crashed, it wouldn’t bring down the SharePoint site; only the sandbox process would crash. So, customer SharePoint sites would stay up, but you’d have a little hole where that custom web part was supposed to be. They thought this would be a fair trade off.

We called these sandboxed solutions partial trust solutions, because this sandboxed process didn’t have all the same capabilities or permissions as the core SharePoint process.

Sandboxed solutions were too restrictive

The sandboxed process was tied to a specific web app and partial trust solutions could only access the site collection they were deployed to. You couldn’t reach across site collections, a common thing developers did in custom full trust solutions.

But even more challenging, was that these partial trust solutions didn’t have full access to the complete SharePoint API; they only had access to a subset of the complete SharePoint API.

This meant that many of the things we used to do as developers in our full trust solutions. So what did customers do? Many of them ignored sandboxed solutions and told Microsoft “go back to the drawing board”.

Blocked many customers from moving to the cloud

SharePoint Server 2010 was the first version that Microsoft used to implement their SharePoint Online (SPO) offering in Office 365. Customers signing up for SPO were limited to only partial trust solutions for any custom code they deployed & subject to the throttling limits imposed on the sandbox.

Microsoft didn’t fully appreciate customer requirements

I think one of the reasons the sandbox and partially trusted solutions were dead-on-arrival when it was introduced was because Microsoft didn’t fully appreciate the problem customers were having. Because they weren’t hosting SharePoint so they couldn’t see not just what types of solutions customers were creating, but how many there were.

When Microsoft extended SharePoint, they used fully trusted solutions. Why? Because they were the only customer and had complete control over SharePoint both in the product that was shipped to customers in SharePoint Server 2010, but also in SharePoint Online.

So, while you can look at the fully trusted solution development model as a first-party (Microsoft) & third-party (customers) development model, the partially trusted solution model was exclusively only a third-party development model.

This is a critical point in the history of the SharePoint Framework as you’ll learn about later in this article and one that I touched on in the first part of this series: What is the SharePoint Framework: SPFx is both for first-party & third-party developers .

Introducing the REST API & CSOM

Before I move on, I want to point out that SharePoint Server 2010 is when Microsoft introduced the beginnings of the SharePoint REST API. This API replaced task-specific ASMX-style web services and opened SharePoint up to a lot more external development off the SharePoint servers. This development happened both in server-side solutions with .NET as well as with client-side solutions in the browser using libraries like jQuery.

They also introduced a client-side SDK, the client side object model (CSOM), that included a .NET SDK, JavaScript SDK, and Silverlight SDK. Today, we only have the .NET SDK implementation which is what we mean when we refer to the CSOM.

Anyway… due to the restrictions I previously explained with partial trust solutions, customers who wanted to customize their SharePoint deployments didn’t move to SPO. They couldn’t address the business requirements that full trust solutions enabled.

Microsoft got the feedback from customers: give us a better option for custom code solutions in SharePoint Online that allow us to meet our needs.

So… yet again, Microsoft needed a better option for developers…

2013 – 2016: Add-in model - SharePoint Server 2013 & SharePoint Online

Microsoft really went back to the drawing board in SharePoint Server 2013. They took a step back and asked themselves: “what do we really need to get developers to do?” The answer: we need to get them off of SharePoint. Specifically, Microsoft needed to get our custom code off of the SharePoint servers.

The ultimate goal was to get our code to run either in the browser or in a process completely separate from SharePoint. But at the same time, they needed to give developers a way to customize SharePoint sites and interact with the data within these sites.

And they came up with a solution…

SharePoint App Add-in model

Microsoft introduced a new way of developing customizations in SharePoint with the SharePoint Server 2013 release. The SharePoint App model, later renamed to the SharePoint Add-in model, enabled developers to create solutions that ran completely external to SharePoint. Custom code ran either in the browser as client-side script (aka: SharePoint Hosted Add-ins) or in some other process (aka: Provider Hosted Add-ins), such as in, but not limited to, an Azure web app.

Apps were given their own identities that you could assign permissions to, just like you can with users.

Significantly improved SharePoint REST API & CSOM

At the same time, Microsoft also made significant investments in the SharePoint REST API and CSOM. They needed to if all these custom solutions were going to get kicked off the SharePoint servers. Developers needed a way to talk to SharePoint.

SharePoint Add-ins were treated as second-class – the ghetto webs

Aside from the steep learning curve associated with developers learning OAuth (a key part of the new add-in model identities) and the REST API, Microsoft also made another curious design decision.

SharePoint Add-ins weren’t added to existing SharePoint sites, rather when they were installed, they were added to subsites that had their own unique URLs. This was done so a custom solution couldn’t easily access the data in the parent site. The label ghetto web started to get popular because these subsites (aka: webs) weren’t treated the same as existing sites.

This frustrated developers and customers because now they were left without a good option to extend existing SharePoint sites.

And as such, customers didn’t adopt the add-in model as much as Microsoft would have hoped.

Microsoft didn’t fully appreciate customer requirements

I’m going to repeat myself a little with the previous sections. I think one of the reasons the add-in model wasn’t adopted by most customers was because Microsoft didn’t fully appreciate the problem customers were having.

Microsoft continued to use fully trusted solutions to extend SharePoint. They didn’t use the add-in model to extend SharePoint. History was repeating itself.

The Add-in model was exclusively only a third-party development model which meant Microsoft didn’t feel the same challenges and pains customers were feeling.

When SharePoint developers couldn’t solve the requirements for their customers, they had to look elsewhere. This time, they didn’t just tell Microsoft the add-in model wasn’t working for them. This time, they went solved the problem on their own with the tools they had available.

2013 – 2016: JavaScript injection - SharePoint Server 2013 & SharePoint Online

No, that’s not a typo on the date. This phase of the SharePoint history overlaps with he Add-in model.

After a bad experience with the sandboxed/partially trusted solutions followed by the add-in model phase, customers started to take things into their own hands.

The add-in model had some significant shortcomings that made it a bad option for many developers. First, the learning curve to working with apps, OAuth, and REST was quite steep for many developers. But more importantly, the add-in model didn’t allow for developers to make changes to existing sites which is what their customers were asking for.

Developers weren’t going to wait around for another solution from Microsoft so they took to the tools that were available to them.

JavaScript Injection

Developers quickly realized that one option available to them to make customizations in SharePoint sites was using client-side technologies.

SharePoint had long included the Content Editor Web Part (CEWP) and more recently, the Script Editor Web Part (SEWP). These two web parts enabled users to add richly formatted content to a page, but more importantly, add JavaScript to the page.

This approach, coupled with leveraging the vastly improved REST API, empowered client-side developers to customize and extend existing SharePoint sites.

JavaScript Injection shortcomings

But, just like all the other development models, there were shortcomings with this option. All customizations were manually deployed. Deployment went something like this:

1. Upload all assets to a document library (JavaScript, CSS, images, etc)
2. Add CEWP to the page
3. Edit CEWP to point to the client side script previously uploaded

The problem with this approach is that anyone with edit rights on the page could change the code of the solution. While this is certainly an advantage to rapid development in many scenarios, it is a problem in organizations where many users rely on a custom solution and someone can easily make a change without going through a change management process or introduces a bug in the solution.

In an organization with a rigorous change management process, these manual steps aren’t the best way to get custom solutions deployed. Nor can a solution like this be easily scriptable to automate deployment of the custom solution.

In addition many customizations employed DOM manipulation, usually with utilities like jQuery, to inject or modify content, functionality, or just to change the page to suit their business requirements.

The problem with this approach is that Microsoft never saw the page’s DOM as an API so they would change it to suit their needs. For instance, maybe a new button was added to the user interface, or the HTML element’s ID changed. If your custom script relies on an element ID that changes, an update to SharePoint Online or SharePoint Server by Microsoft can break your solution with no warning.

What’s next for SharePoint developers?

After the SharePoint 2013 release, SharePoint engineering started looking at the SharePoint development landscape. They were evaluating what developers were using, were not using, and balance those with the current goals of the platform.

JavaScript Injection dominates

One observation they had was that the JavaScript injection technique of creating solutions had become very popular by customers.

The CEWP & SEWP were, by far, the most used web parts in SharePoint Online as they were usually the way people deployed their JavaScript injection-based customizations. At the same time, they recognized the shortcomings of this customization approach.

Promoting client-side development to a first class option

Microsoft wanted to take this approach customers were clearly already doing, and not only adopt it, but improve upon it.

This new approach not only addressed many of the goals & needs of SharePoint, but also of customers.

• Custom solutions would be completely decoupled from the SharePoint process as execute entirely in the client (browser). Customizations are secure in that they only run in the context of the current user.
• Customers can apply customizations to existing sites through simply installing an app.
• Solutions are easier to create and deploy in that they don’t need to be hosted external to SharePoint. Solutions are deployed to SharePoint and while the files are stored in SharePoint, they are only executed in the browser.
• Establish an API (contract) on areas of the page that can be customized.

Conclusion

And that is where we are today. This new approach that Microsoft came up with in 2016 is the SharePoint Framework.

My goal in this post was to provide some context and explain the developer history of SharePoint. Hopefully now that you have some context, you see the following:

As SharePoint evolved over time into the product we have today, one that is primarily a hosted implementation in SharePoint Online, Microsoft iterated through multiple development models until they found one that worked for both developers & Microsoft alike. They came to this conclusion by addressing the challenges and requires they were faced and by observing what customers were doing through their implicit actions.

This is one of a multi-part series of posts answering the most common questions on the SharePoint Framework. This series, SharePoint Framework Five “W"s & One “H” answered , gives you the best high-level picture of the SharePoint Framework. I ( Andrew Connell ) wrote this series in late 2020 to help people new to the SharePoint Framework get the answers to the most common and basic questions about the SharePoint Framework.