• Category Archives Custom Content Type
  • NT Authentication Failure: Utilizing Custom Web Services External to SharePoint

    Scene:
    We have a server hosting a webservice (we’ll call this the Sending Webservice) that takes documents/pdfs, etc. and uploads them to a specific site collection in SharePoint. On the SharePoint WFE, there is also a custom webservice (we’ll call this the Receiving Webservice) running in the same app pool as the hosted SharePoint site collection that receives the authentication, connection and then uploads the document into the document library.

    This is where the fun begins. Our security department is running the domain controllers on Server 2003. OOTB Server 2003 comes with an ancient credentialing capability called LanMan or Lan Manager. I am no security guru, but I’ll explain it’s purpose as best I can.

    From my understanding, the Lan Manager attaches a hash to every account that has a password tied to it within active directory. This hash could be used by applications to pass an encrypted password through to A.D. and obtain a successful authentication. The encryption methods this hash could handle were for both LM and NTLMv1 (version 1) encryption. Both versions were incredibly weak and easy to hack if someone was sniffing the network traffic watching for this handshake. LM and NTLMv1 have also been retired for over 10 years. You do not need to fear if you are running domain controllers with Server 2008 or greater as Lan Manager is all but gone from this environment now. But by default, Lan Manager and the LM authentication is enabled when running Active Directory in Server 2003. However, you can easily disable these.

    Back to the story. Our security department decided to do several things to increase security. Change the user ids to unrecognizable letter and number combonations and force all passwords to 15 characters with all the standard character requirements that would go with it. They didn’t mention Lan Man had already been disabled the week earlier. Now here is the catch…once the Lan Manager was disabled, the hash remained tied to the account. So the service account that was using an “LM” or “LMNTv1” encryption hash still worked just fine. But once a password or ID on the account was changed, the hash that was originally tied to the account in Active Directory dropped off and the account failed to authenticate with these encryption methods.

    A good test to see if this is ever the case in another situation, change the service account to an ID that did not have the password changed after Lan Man was disabled. If it’s a “Lan Man” issue, the account will work just fine. Then change the password and expect to see the functionality break again.

    As we had already recieved the updated account credentials and they were not working, we reverted the account back to the original service account login ID, display name and password credentials just to make sure we wern’t missing anything on the code side that was holding the credentials somewhere we did not realize…but even with the reverted credentials it was failing.

    When we look at the log files, it gives the error that the credentials supplied were invalid. That’s pretty much it. This did not anser the really important questions. Such as, “what is requesting the credential (AD or SharePoint)? What is sending the credential (recieving or sending webservices)? And who is saying it is denied (SharePoint or AD)?” Now, we know the account worked just fine if you used the account via the front end of SharePoint so we know it is not end user error. You could log into a site just fine. But using the Sending wservice caused the problem. Is SharePoint recieving a hashed password and rejecting it automatically? or was was it capturing the authentication and sending it re-encrypted to Active Directory and then AD was denying it because of that or was SharePoint simply passing the authentication through (sending on whatever it recieves).

    A lot of questions that we couldn’t answer without some help. So, we got our Network guru on a call and he started to watch the traffic coming from and going out from the SharePoint WFE server (I believe with a program called NetMon). We triggered the Sending webservice. There we saw the credential go to the WFE and pass on to the domain controller. Then the DC rejection packet. Both the Sharepoint server and the AD server rejected the credential.

    This answered, who was doing the rejecting (both SharePoint and AD). Which also means that the credentials were not geting reencrypted by the SharePoint server since SharePoint itself was rejecting the credential.

    After many hours trying to isolate the issue, we learned that there was another Sending webservice that was using this custom SharePoint Receiving webservice and it was working just fine. :-/

    So in the end, when an account is requesting to get authenticated to SharePoint, SharePoint does nothing but try to authenticate it itself. And if it cannot, SP will pass on the credentials (as it received them) to Active Directory for authentication. AD says yeay or nay, and sends that answer back to SharePoint. SharePoint then simply allows or denies based on that response from AD.

    Below is a depiction of the the path security takes:
    1) Client pc requests action from the hosted webservice.
    2) Webservice passes it’s service account credentials to the custom webservice hosted on the SharePoint server.
    3) SharePoint receives credentials and attempts to authenticate the account itself. If it cannot, it passes on the credentials to the Domain Controller for authentication within Active Directory (AD).
    4) AD approves/denies credential and sends response back to SharePoint WFE requester.
    5) SharePoint notifies original sender of the response.
    The Credentialing Process
    NTLM Path
    Useful information:
    http://www.techrepublic.com/article/tech-tip-lock-down-systems-by-disabling-lm-authentication/5287636



  • Create a Custom Content Type Lookup Field that Pulls Data From External (SQL) Custom List.

    Creating a custom content type that is a lookup to an external content list is not as straight forward as all that. But once you have completely built one, it’s significantly easier than resources on the web would have you believe. The only downside is there are A LOT of steps.

    We had a scenario where we had several Custom Content Types (CCT). Each of the CCTs needed a custom site column look up field that pulled data from a SQL database. A fairly common business requirement but quite tasking to complete from scratch. As I’m a big proponent of visual aids, I’ll include as many pictures as possible and keep the instructions to a minimum but clear.

    The task consists of the following steps:

    • Secure Store Service configuration
    • SP Designer – Create (ECT), read item & read List & custom external list
    • BDC Service App (permissions)
    • Add to a custom list to test the ECT functionality
    • Verify the new list in the hosted site.
    • Create Custom Site Column

    Secure Store Service (SSS)

    In this situation, we need to pass a security account to the SQL database. This is where the SSS comes into play.

    NOTE: Before beginning, this is assuming you already have an account configured on the SQL side that has the needed access to the data you are querying for.
    Access the SSS in Central Administration > Application Management > Secure Store Service
    1) Create a secure store item by selecting ‘New’.

    2)     In the next window complete the fields below and select ‘next’.

    3) The next steps are titles for the columns for storing the credentials. NOTE: that this is not the credentials themselves.

    4) You can setup custom permissions for a specific set of users to be able to manage this SS item. Select OK.

     

    5)     Enable the checkbox and hit the “Set” button to add the credentials.

     

    6)     Enter the owner of the credentials and the account credentials. Hit OK.

     

    The SS account has now been created and is ready to be utilized in the setup of the External Content Type we will create in SharePoint Designer.

    *Note: If this is to be setup on a farm, the Secure Store Service will need to be enabled on each server hosting a web interface that will utilize this account.

     

    External Content Type Creation & Configuration

    Now we are going to create the connection that pulls the data from the database into an external SharePoint list.

    1)     Open SharePoint and connect to the site you want to build out your connection to.

    2)     Select ‘External Content Types’ from the Site Navigation window and choose the ‘External Content Type’ button in the top left corner of the ribbon.

    3)  Complete the following fields: Name & Display Name. Then click the link ‘Click here to discover external data sources and define operations’.

    4)    On the next screen, select “Add Connection”. Since our connection is SQL based, we will choose SQL from the drop down.

    5)     Next, complete all the fields, and refer back to what your original SSS Identity was for the Secure Store App ID:

    6)    Now we drill into the database for the table we need to pull data from & create the needed ECT Operations. We do this by locating the respective table, R+Click and choose the needed option. I only need READ access, so I will choose Read of both Item & List options. I will choose Item first.

    *note: At minimum a Read Item and a Read List must be created and configured before this ECT can be saved or an external list created.

    7)    The Operation and Display names are inconsequential to the purpose of my needed functionality. So we will leave these as they are. Click Next.

    8)     The next window shows the Input parameters. We only need a single column to map as an identifier for the columns we are tying to the list we are building. Hit ‘Next’.

    9)     Here we need to make sure we map at least one identifier and check all the fields that we wish to include in the Custom list. Click Finish.

    NOTE: The display name field is the name that will get translated to the column of the external table. So make sure this is a legible name for you & your end users instead of a naming standard for your SQL data column.

    10)     Next is to create the Read List. R+Click on the column again and choose the Read List Operation.

     

    11)     Again, hit Next. Here you can include filter parameters if you like. At this time, I do not need them. There are plenty of good resources that can walk you through this process. Hit Next again.

     

    12)     Here in the Return Parameter Configuration we will do the same thing we did in the Read Item Configuration. You also have an option called “Show in Picker”. Checking this will enable the specified column to be displayed in our lookup column (picker content type). We will want to check this for the needed columns pulled from the table. Hit Finish.

    13)     We now need to create the Custom List on the SP site. Hit the Create Lists & Forms button in the ribbon and follow the instructions for the list configuration.

    1)     Once the external list is created, you can view it on the site you connected this to. I would suggest gonig to the site and make sure the data is pulling in properly. At this point, you are now ready to create the custom lookup content type.

    2)     In the site, go to Site Actions > Site Settings > under Galleries, select Site columns.

    3)     Select “Create” at the top of the page.  From here the rest is pretty self-explanatory. Choose the “Lookup” information type and the lower column settings allow you to select the newly created external content list and from all associated columns from within that list. Once you save the custom Site Column, you can now add it to any library/list you wish.

    *NOTE: There is another option at the base of the lookup content type that allows you to add additional columns that are associated with the external content list. This feature, though offered, is unfortunately NOT supported and does not work. Even if you try to build it out programmatically.


    Links to some information I found useful in completing my task:

    http://msdn.microsoft.com/en-us/library/ee557243.aspx

    http://msdn.microsoft.com/en-us/library/ee558778.aspx

    http://answers.flyppdevportal.com/categories/sharepoint2010/sharepoint2010general.aspx?ID=7c47c583-ff7a-4e99-b221-9da1ee0c8517

    https://littletalk.wordpress.com/2011/08/18/external-content-type-an-error-occurred-while-retrieving-data-from-a-system-administrators-see-the-server-log-for-more-information/

    http://msdn.microsoft.com/en-us/library/ff798274.aspx