My Queue is Stuck! – How to manage your queue service in Project Server 2007


The new world of Project Server 2007 and the architectural changes are catching a few of our customers out – and I thought I’d share a few tips and tricks for keeping the queue flowing – and some tips for getting things moving again if they appear to have stopped. 

First I will point to a great TechNet article on the Queue (*** 12/18/2014 update – try this link now http://technet.microsoft.com/en-us/library/cc197395(v=office.12).aspx ***) and as you will all have read this then my explanations will make more sense :).

Under Server Settings in Project Web Access the Manage Queue option allows you to see what is happening in the project and timesheet queues – if you don’t have admin access then the Personal Settings will give you a glimpse of your queue jobs.  The latter option may not however give you the complete picture and allow you to see what might be ahead or you.  It is like being stuck on the highway and not being able to see around the corner to where the flashing lights are… 

So lets start with some definitions:-

Waiting to be processed – means exactly what it says.  Once I get to the front of the queue then I am ready to go.  But there may be other active jobs ahead that will stop my job starting even if I am first in line.  The queue is clever enough that it will hold jobs back if their processing would interfere with other running jobs.  An example might be a publish job that will need to wait for a cube build to finish.

Processing – means that I made it to the front of the queue, was allocated a thread and am working away!  One thing I have noticed is that the % complete indicator doesn’t always make you think that “processing” is happening – but generally it is.  Looking in the ULS logs, event logs or at general server activity (particularly the Microsoft.Office.Project.Server.Queuing.exe process should help if you have continued doubts that processing is moving along.

Skipped for optimization - is the queue’s way of telling you that it is not going to do the same thing twice.  Some queue jobs have a payload (such as saving a project) and others are merely instructions (such as publish a project).  If several of the same instruction are in the queue, then only one needs to be actioned.  An example might be working on a project and publishing a few times during a period of time.  If the queue was busy all of these jobs might be sitting waiting for a while – and then rather than doing each in turn it just needs to do one.  It is just an instruction to publish the content of the saved project.  This would not happen with a queue job that had a payload as each of these contains real data that needs to be applied – rather than just an instruction to do something with data somewhere else.

Getting Queued - appears to be one of the more confusing messages.  I mentioned above that some jobs, such as save project from Project Professional, have a payload. This payload goes into the queue as a group of related messages, which then get processed once they reach the front of the queue.  Getting queued means that these messages are going into the queue.  It is possible that the Getting Queued message appears for some time because a very large project is coming in across a very slow link.  One other potential problem that can break things is if this flow in of messages does not complete.  Perhaps the Project Manager saving the project shuts down Project before it completes – or perhaps goes out of wireless range midway through the process.  Either way the Getting Queued could sit there for some time. To fix this up find the person who has this project in mid-save and get them to reconnect and complete the job.  As a last resort you can cancel the Getting Queued – but YOU WILL LOSE DATA!  Any changes the Project Manager made will not get saved.  To protect you from inadvertently canceling one of these jobs we add a check box under Advanced options labeled “Cancel jobs getting enqueued” which will need to be checked before these jobs can be canceled.

Failed and Not Blocking correlation – is a failure that is isolated and not stopping any other jobs from processing.  The term correlation is used to group related queue jobs together.  There should be an associated error message and entries in the log to help explain the problem.

Failed and Blocking correlation – means that something bad happened that is also blocking other things in the related group.  If a save fails then a publish could not continue would be one example.

Success – is the one message we like to see!  It can also be useful to sometimes show the Success messages (by default they are not shown in the Manage Queue display) as it is a way of seeing if the queue is working at all.  Adding the completion state of Success through the options on the manage queue page is how this is done.

Canceled – means what it says.  It could have been canceled by a user, but it is also possible for jobs to be canceled by the server.  One example would be a failure early on in a save from Project Professional.  A job would have been added to the queue for the save – but reconnection may lead to cancellation of this job and the addition of another save job – it really depends hoe far the save got before the problem.  I simulate bad things like this by pulling my network cable out just after hitting save – just to see what happens!

I will follow up with another posting on the queue with some further tips on troubleshooting -but my parting gift is a guide to what the dialogs at the bottom of Project Professional 2007 mean during a save.

    • Blue progress bar- saving to local cache
    • Synchronizing data to server…- The data is going from the local cache to the PSI and being passed into the queue (Getting Queued)
    • Save job xx% complete.  Expected Wait Time 20s - The job is either Waiting to be processed or more likely Processing.  Once you see this then it is safe to close Project Professional – your saves are safely in the queue!

Technorati Tags: Project Server 2007


Comments (106)

  1. Sushant Srivastava says:

    Very Helpful. Would be great to have a list of best practices to avoid the queues to get blocked.

    Thanks! bgcoepjs@microsoft.com

  2. Pisir says:

    It’s helpful. I’m a project administrator. I am having a problem wiht the queue system. In the Manage Queue page, there are always a lot of items with the status Waiting to be processed. But none item finished no matter how long time I wait. This case happens day after day.

    I think I need your help if you like to. Thanks very much!it@tradinasia.com

  3. BriSmith says:

    There is usually a good reason jobs are waiting to be processed.  Make sure you set the history setting on the page to see old jobs (say from 1/1/2007) and hopefully this will identify what is making the jobs wait.

  4. bchild says:

    Your site is defintly the best on the net for fixing common Project server issues (BIG THANK YOU!) However I am constantly fighting the queue service.

    Timesheets will get stuck and never process the only way to clean out the issue is to go into services and re-start the service! This can happen anywere from once a month to multiple times a day.  Very frustrating when you are trying to do payroll.

    Thank you for any help you can provide.

  5. BriSmith says:

    Hi,

    If re-starting the queue service fixes things then this doesn’t sound data related, as once bad data is in the queue it will just block things and generally even a reboot wont fix things.  So it appears you queue is stopping and not getting re-started.  I would check the event log ofr any errors that may indicate the failure.  Perhaps setting up a scheduled stop/start of the service using the net stop command might help (the service name can be identified throuh the services mmc plug in).  If you open a support incident then I am sure my colleagues would be able to get to the bottom of the problem.

    Best regards,

    Brian.

  6. alexD says:

    hi, pls tell me how can i restart the service and what services must to restart? thx a lot

  7. BriSmith says:

    The services can be re-started either from the Administrative Tools, Services applet (look for Microsoft Office Project Server Queue Services)or at a command prompt using:-

    net stop ProjectQueueService

    net start ProjectQueueService

  8. Ron says:

    Hi Brian,

    Do you happen to be aware of the following type of log messages?

    “PWA: http://epm2007/pwa, SSP:SharedServices1, User: wssadmin, PSI: [QUEUE] ProjectQ: Group is sleepting: d5122… type: ReportResourceSync 123e…”

    I couldn’t clear the queue even by restarting the Project Queue Service or rebooting the project server. By the way, the database is on a separate server.

    Any suggestions?

    Regards

    Ron

  9. bchild says:

    Brian,

    I would be happy to open an incident can you direct me to whom I should contact to open a case?

    -Brian

  10. BriSmith says:

    Hi Brian – there are various ways depending on any contract you might have through your company.  Best way is to take a look at http://support.microsoft.com/oas/default.aspx?gprid=11388.  

    Best regards,

    Brian.

  11. marcopf says:

    Hi Brian, i have four projects stucked in the queue. When i try to force check-in of these projects, nothing happens. What can i do????

  12. BriSmith says:

    Hi Marco,

    Usually projects appear stuck because something is blocking them in the queue from some time in the past.  Change the history setting in the queue to look at really old jobs and you may find ones relating to those projects.  When you do then these need deleting (and possibly using the "Cancel jobs getting enqueued" option mentioned in the main article.)  Force check in will just add another check in request to the queue so it isn’t very "forceful".  If this does not work for you then I would suggest a support incident be raised with Microsoft.

    Best regards,

    Brian.

  13. Mike says:

    We keep having the same issue. After a user checks out a particualar project, then “saves” the Project Save from Project Professional stalls in the Getting Queued, and never completes. Which stops the project from being checked in. I have started and stopped the queue service but it doesn’t seem to have an effect. This only happens on one project. This only seems to happen to the user. I have performed the same set of steps from my machine, using the same version of project professional 2007 and had no issues. Any ideas?

  14. BriSmith says:

    Hi Mike,

    The most likely reason is that this particular user is just closing Project Professional before the save gets far enough – so some of the data hasn’t made it back to the server.  This is a bug we are looking at – but by use of the file save, then waiting for the status message at the bottom to show completion, then file close, and waiting again for check-in and finally File, Exit – should give a better experience.  I would expect other errors if theere was a data problem.  Stopping and starting the queue service (as long as it was running in the first place) will rarely fix anything as the queue is data driven – and if the data doens’t change (e.g. completion of the save) then neither should the behavior of the queue’s action.

    I hope this helps.

    Best regards,

    Brian.

  15. Elad says:

    How do I handle this failure ?

    It is happanning only with one project of mine …

  16. Adrian Bear (SDM) says:

    Hi Brian,

    Excellent content here Brian. This site is often my first port of call.

    I have a client who has created a Test Project Server environment by taking a backup from Production and restoring it.

    Unfortunately their Job queue never gets processed in this Test environment. I can cancel jobs and add jobs (e.g. Saving a project) but all jobs end up as “Waiting to be processed”

    I have been reading the MS article you have linked to above and been thinking about the following statement.

    “Every job polling thread looks for jobs originating from a specific instance”

    Does this mean the Queue Thread will not process any jobs because it is looking for jobs from the “Production” instance of PWA and not the “Test” instance?

    Adrian.

  17. BriSmith says:

    Hi Jason,

    The document at http://technet2.microsoft.com/Office/en-us/library/348d1f05-5cc6-41bb-be2c-ea28bbf1c3421033.mspx?mfr=true may help understand what might be wrong.  Possibly the queue service didn;t start correctly.  You should see 1 queue service process, plus another for each SSP that has PWA sites configured.  So usually a minimum of 2.  If you just see one then try re-starting the service.  It can fail to start in some configurations if SQL Server is not accepting connections at the point the queue service starts – I often see this behavior in virtual or single server systems.

    It should find any instance of pwa for processing – even in a system restored from another server.

    Best regards,

    Brian.

  18. Bohdan says:

    Hi, Brian

    We also have a problem with queue stuck. After small investigation of log we saw the following message:

    [RDS] The RDS message will go to sleep because a RDB Refresh is in progress.

    As we understood Reporting Database is in the middle of refresh but it is not true. There is no jobs in queue with type “Reporting Database Refresh” all stucked jobs are of type beginning with “Reporting…” for example (Reporting (Project Publish), Reporting (Resource Sync), Reporting (Lookup Table Sync)). Our PS2007 installed in stand-alone mode (all on 1 computer). Where can be the problem?

    P.S. By the way Reporting DB is almost empty (no projects, no tasks etc).

  19. Tom Xie says:

    I had a problem with Project just before this that caused me to kill the MS Project application and restart. Shortly after that, I re-started MS project, re-opened the project and then saved and published.

    Then a mail came with the subject “Your queue job ReportingProjectPublish failed. Please contact your administrator for assistance”, then I checked the queue, I find the type of the queue of the project showed:Failed But Not Blocking Correlation. What causes this issue? What shall I do to prevent it from happening? What shall I do with the queue? retry or cancel?

  20. BriSmith says:

    Hi Tom,

    I can’t say what caused the crash of Project, but this could certainly upset the queue if something is in the middle of precessing.  I would try "retry" first, and if it still fails then just "cancel" this job and then re-publish the project and see if all goes through the next time.

    Brian.

  21. BriSmith says:

    Hi Bohdan,

    Sorry for the delayed response. The reporting database refresh can take some time – and certainly if for some reason it failed (such as rebooting the server) then it may never finish but still think it is running.  I would suggest opening a support incident to dig deeper into this issue.  For options see http://support.microsoft.com.

    Brian.

  22. Anupom says:

    My environment is a ‘single server’ installation. Everything was going smooth initially ( saving, publishing, cube generation, etc). but now all jobs go into ‘waiting to be processed’ state and never beyond.

    No where it gives any error/warning message. I have checked in all log the files. Can anyone put some light on this issue ?

  23. BriSmith says:

    Hi Anup,

    First stop is to check your queue service is running – and it isn;t enough to just look in services (but that is a good place to start).  If you open Task Manager on your application server you should see 2 processes for queue and 2 for event service (or more if you have more than 1 SSP).  If you just see one try restarting both services – they may have timed out when last started contacting SQL Server.  Also if SQL has been down and up they may need re-starting just to re-gain the connection (this last piece is fixed in SP1).

    Let me know how this goes.

    Brian.

  24. Ben says:

    Not sure if this is a red herring for my issue but I get ‘Skipped For Optimization’ in the Queue whenever I do the below:

    My issue is approved tasks are not updating the project Plan. If I create an Activity and assign a task to a team member, the user changes the percentage complete for the task and the task gets Approved, we are not seeing the update on the Activity. The ‘Preview’ shows the update to the Activity Plan, but the actual Activity does not get updated. This only thing I see logged is ‘Skipped For Optimization’ in the queue.

    Any ideas anyone?

  25. Ben says:

    Sorry to waste your time Brian, the Activites we were trying to update were checked out, so not receiving the task updates. You can cancel my last blog comment.

    Keep up the good work, your site is invaluable!

  26. BriSmith says:

    Hi Ben,

    I’ve kept your comments on here so that the answer you found might help someone else.  Generally the skipped for optimization means that there is another job in the queue that will do the same thing – so it doesn’t need to process both.  So for every skipped for optimization I would expect to see a success at around the same time.

    Brian.

  27. Anupom says:

    Hi Brian,

    Thanks for quick response and I am sorry for this delay from my side !

    I observed, under ‘Process’ tab of ‘Task Manager’  only 1 process for queue and 1 process for event service are running on the application server. I have tried restarting both the services but with no difference.

    While I try ‘New Task’ option to start any of those services from the ‘Task Manager ‘(Applications), it throws a message saying "Cannot start services from command line or a debugger. A Windows Service must be installed and then started with the ServerExplorer, Win services Admin tool or the Net Start command"

    How do I make 2 processes for each queue and event services running ? Is there any Win service missing on the server ? Please help.

    Thanks

    Anupom

  28. BriSmith says:

    Hi Anup,

    I would look in the ULS logs.  It should be giving some indication there of why the second instance isn’t getting started.  You cannot start this manually.  When you start the service the service itself will check the database to see which SSPs have Project instances provisioned and will start this 2nd instance (and 3rd, 4th if necessary). If it has a problem I would expect this to get logged.  As another test you could try creating another SSP and another PWA site.

    Brian.

  29. SikRip says:

    Hello,

    I have a job with “Failed and not blocking correlation” but it seems that in fact it does block any new job…

    How can i cancel or delete this queued job?

    Thanks

  30. BriSmith says:

    Hi SikRip,

    If it says it isn’t blocking then this is usually very reliable.  Take a look for other, perhaps older jobs.  You should always use Retry before thinking about cancelling – and also you do not say what type of job it is – so I wouldn’t like to give guidance without more details.

    Brian.

  31. Devender says:

    I tried to build the cube,but my jobs are getting stuck in the queue.i even tried to force checkin the projects it gives me again the same state

    then i cancelled all the jobs in the queue and even restarted the service but still the queue is stuck.

  32. BriSmith says:

    Hi Devender,

    I would check the ULS logs.  If the cube job was waiting in the queue then something was stopping it.  First check the queue is running (see other postings on this) but if it is then you need to fix what is blocking this specific job.  Any amount of stopping and starting services isn’tr going to change the data that is driving the queue.

    Brian.

  33. mikey says:

    Hello!

    I have an annoyng problem with tasks publishing operation. I create a new task, I approve it, but the changes are not visible in Project Center. The only solution is to open the project with Project Proffesional and publish the project from there and in my scenario this is not possible.

    Every time I approve the task the job fails (Failed but not blocking correlation) with the following error message:  

    Error summary/areas:

    Reporting message processor failed

    ReportingProjectChangeMessageFailed

    ReportingProjectChangeMessageFailed

    ReportingProjectChangeMessageFailed

    ReportingProjectChangeMessageFailed

    ReportingProjectChangeMessageFailed

    ReportingProjectChangeMessageFailed

    Queue

    GeneralQueueJobFailed

    Error details:

    <?xml version="1.0" encoding="utf-16"?>

    <errinfo>

     <general>

       <class name="Reporting message processor failed">

         <error id="24006" name="ReportingProjectChangeMessageFailed" uid="9ce09def-e83c-4c57-a965-ddec33fa532f" QueueMessageBody="Project UID=’1e5ae5f8-863c-4a7d-adc7-68300b338516′. PublishType=’ProjectPublish’" Error="Violation of PRIMARY KEY constraint ‘PK_MSP_EpmTask’. Cannot insert duplicate key in object ‘dbo.MSP_EpmTask’.&#xD;&#xA;The statement has been terminated." />

         <error id="24006" name="ReportingProjectChangeMessageFailed" uid="78bac897-e890-438e-87e2-d6bd11692557" QueueMessageBody="Project UID=’1e5ae5f8-863c-4a7d-adc7-68300b338516′. PublishType=’ProjectPublish’" Error="Cannot transfer project : ‘1e5ae5f8-863c-4a7d-adc7-68300b338516′. A FK constraint was thrown." />

         <error id="24006" name="ReportingProjectChangeMessageFailed" uid="f46e6d07-1a03-4b17-8b4b-8a4e9252a933" QueueMessageBody="Project UID=’1e5ae5f8-863c-4a7d-adc7-68300b338516′. PublishType=’ProjectPublish’" Error="Cannot transfer project : ‘1e5ae5f8-863c-4a7d-adc7-68300b338516′. A FK constraint was thrown." />

         <error id="24006" name="ReportingProjectChangeMessageFailed" uid="7421949b-0ceb-4f7f-a7ea-87e118ca3b63" QueueMessageBody="Project UID=’1e5ae5f8-863c-4a7d-adc7-68300b338516′. PublishType=’ProjectPublish’" Error="Cannot transfer project : ‘1e5ae5f8-863c-4a7d-adc7-68300b338516′. A FK constraint was thrown." />

         <error id="24006" name="ReportingProjectChangeMessageFailed" uid="cacd3b25-fa97-4831-beaf-a8876ffeeb05" QueueMessageBody="Project UID=’1e5ae5f8-863c-4a7d-adc7-68300b338516′. PublishType=’ProjectPublish’" Error="Cannot transfer project : ‘1e5ae5f8-863c-4a7d-adc7-68300b338516′. A FK constraint was thrown." />

         <error id="24006" name="ReportingProjectChangeMessageFailed" uid="fa15a345-96c4-4bb2-b4ed-06c7f9f11602" QueueMessageBody="Project UID=’1e5ae5f8-863c-4a7d-adc7-68300b338516′. PublishType=’ProjectPublish’" Error="Cannot transfer project : ‘1e5ae5f8-863c-4a7d-adc7-68300b338516′. A FK constraint was thrown." />

       </class>

       <class name="Queue">

         <error id="26000" name="GeneralQueueJobFailed" uid="8ad9ba5e-54ec-4d67-8fea-7b770fc1694b" JobUID="eaba023e-60b2-47e3-ac38-cb6be8bb1349" ComputerName="PROJECT07" GroupType="ReportingProjectPublish" MessageType="ReportProjectPublishMessageEx" MessageId="1" Stage="" />

       </class>

     </general>

    </errinfo>

    Is there any way to solve this issue?

  34. Anupom says:

    hi Brian,

    Thanks for your response ! The issue stands unresolved still. I tried to create a new SSP to build another PWA site, but failed. It throws “Access denied”. Looks like some n/w policies are affecting the setup. However, the logs do not say anything.

    -anupom

  35. BriSmith says:

    I think you are on the right track with permissions.  Check that all the service accounts and the account you are using have correct access.  Do the logs really say nothing?  Perhaps this is a permission issue too?

  36. Locus says:

    Hi Brian, we have tried the range of measures described in the blog to get a project published, but cannot get it through. After checking right through the log as far back as we can it appears that the original publish was terminated at this level.

    What are the best steps for recuperating the file and retaining the data, and what is best practice if you cannot ‘save’ the file?

  37. Harlow Burgess says:

    After a fresh install of Project Server 2007 and SP1, nothing in my queue ever got processed. I found that the queue service was not running. I tried to start it and it failed.

    “The Microsoft Office Project Server Queue Service service failed to start due to the following error:

    The service did not respond to the start or control request in a timely fashion.”

    I tried various service accounts, from a domain account to Local System and nothing worked. Have you seen this before???

  38. BriSmith says:

    I think you are seeing this issue – it also needs a restart to become effective.

    It may be a time out issue getting a response from the Service Control manager.  One workaround my Colleague Eric Marinho da Silva found is as follows.  It involves making changes to the regsitry so only try this if you have backups of your registry and know what you are doing:-

    When we start to run the Queue or Event service we have the following event ids:

    7009

    Service control manager

    Timeout (30000 milliseconds) waiting for the     to connect.

    7000

    Service control manager

    The service did not respond to the start or control request in a timely fashion

    To fix the issue, please use the following steps:

    1. Run "C:Program FilesCommon FilesMicrosoft SharedWeb Server Extensions12binpsconfig –cmd upgrade –inplace b2b –wait –force".

    2. In registry editor add/modify the following values on "HKLMSystemCurrentControlSetControl":

    a) ServicesPipeTimeout   DWORD   60000  (decimal).

    b) WaitToKillServiceTimeout changed to “120000”.

    3. Reboot the server

    Please let me know if this works for you.

  39. Harlow Burgess says:

    That worked. The service is running and my queue has been cleared. Thank you!

    FYI – I am running App and Web together on a virtual server with a separate SQL Server backend. The VM has 2GB RAM and is running on a physical server with Microsoft Virtual Server 2005 R2 SP1 on Intel VT. The SQL Server is an 8 core server with 32GB RAM and fibre channel SAN.

  40. Scott Stumpf says:

    Bohdan

    Hope you are still looking at this message thread.

    I have discovered a fix for the sleeping thread issue where you see

    “[RDS] The RDS message will go to sleep because a RDB Refresh is in progress.”

    in the SharePoint log.

    Basically, Project Server is stuck in a mode where it thinks a reporting database refresh is in progress and other jobs are getting blocked.  I’m not sure how you get into this state; possibly a hard server restart in the middle of an update.  You can go into the reporting database using SQL Server Management Studio and call the stored procedure MSP_RefreshModeCompleted.  Here’s the SQL statements

    DECLARE @RC int

    DECLARE @AdminItemTypeUID uniqueidentifier

    DECLARE @RefreshStatus int

    SET @AdminItemTypeUID = ‘c795e698-6248-4fa4-a788-6b605eedcfa6′

    SET @RefreshStatus = 0

    EXECUTE @RC = [ProjectServer_Reporting].[dbo].[MSP_RefreshModeCompleted]

      @AdminItemTypeUID

     ,@RefreshStatus

    Once this procedure is executed, values in the MSP_AdminStatus table will be updated.  The next time the job queue checks the status it will be able to complete the queued reporting jobs.

  41. BriSmith says:

    Hi Scott,

    Although I am usually against any direct database changes you are spot on with this posting – and the only thing I would add is for anyone using this script to be absolutly certain this is their scenario and that the reporting database refresh is not still running before trying this “fix”.  

     Thanks for sharing!

    Best regards,

    Brian.

  42. In reply to the posting of Mikey on the 1st of Feb. 2008:

    Hello,

    We are experiencing the same kind of problems you state with several or our

    projects aswell. We are still waiting for an implementation of SP1 by our IT

    department, so can’t confirm this would fix it.

    Here is a part of the string we get on one of the failing project we intent

    to publish to the reporting database:

    <error id=”24006″ name=”ReportingProjectChangeMessageFailed”

    uid=”b0f26439-385b-4529-ac3f-e3c5edc1afde” QueueMessageBody=”Project

    UID=’bfd73c23-3d2b-4149-8b92-30d77f8f96bb’. PublishType=’ProjectPublish'”

    Error=”Cannot transfer project : ‘bfd73c23-3d2b-4149-8b92-30d77f8f96bb’. A FK

    constraint was thrown.” />

    Have you found out how to fix or resolve these kind of problems? This is a

    big problem as our projects have got Actual Work hours within the

    project-files, but in the published database these hours keep on growing and

    in the reporting database this proces has come to a standstill… so making

    reports with Data Analysis seems to be worthless now.

    Hope you know how to help us.

    Kind Regards,

    Wijgert IJlst

  43. In addition to the above, I also have noticed that the concerning projects (which seem not to be published in reporting db again) are missing in the reports coming from the OLAP-Cube/Data Analysis.

    In Project Server (publish db) the current data from the concerning projects are available to (re)view.

    So now not only the data is failing to update into the reporting db, but also the older data is missing.

    What should I do to resolve this problem?

  44. BriSmith says:

    Hi Wijgert,

    It would be expected that you wouldn’t see these in the cubes if they are not getting to the reporting db.  The error says FK problem which stands for foreign key.  I suspect a custom field or possibly even a resource that is referenced in the project failed to get to the reporting database.  Try opening and saving all lookup tables, then the same for custom fields (which will republish) and then use ProjTool from the SDK to re-publish – using the full publish option.  Check the queue frequently and address any errors that occur.  I can’t say how to address as I don’t know what the errors might be :).

    Best regards,

    Brian.

  45. Rob says:

    Hi Brian,

    So when you say "get them to reconnect and complete the job". Do you mean just have them open Project Pro and connect or do they have to connect and open the project schedule?

  46. BriSmith says:

    Hi Rob,

    If they just open Pro and connect to the same server with the same cache then any incomplete jobs should continue – assuming the cache hasn’t been cleared and the queue job is still sitting there.  Be aware that the cache if opening the project directly from Pro is different to that used if the project is opened indirectly from  PWA.

    It depends exactly what point it got to if it will continue – you may see a cancelled job and then a new save begin.

    Best regards,

    Brian.

  47. Michelle says:

    Hi Brian:

    We have am issue that I haven’t seen before.  The project server app is on one box and the sql server 2005 is on another, a virtualized server.  Whenever we try to publish, save or do anything, the queue always says ‘Waiting to be processed (Sleeping).’  This is during UAT, so there are about 2 people total on the system.  We have applied SP1 and the recent Infrastructure upgrade.  They are not using MOSS, only WSS.  Any ideas as to why every event in the queue is ‘sleeping’?  

    Thanks, Brian!

    Michelle

  48. BriSmith says:

    Hi Michelle,

    Usually you will find clues to "sleeping" jobs in the ULS logs, where it may give details of why the job went into a sleep state.  It usually does this if something else is happening that it needs to wait for – and it will wake up every 5 minutes or so to see if the condition has changed.  It is possible it thinks something such as a reporting database refresh is happening when in fact it isn’t (or it was –  but has terminated unexpectedly, which could be due to a number of reasons).  As there could be a number of different causes you may need to open a support incident to have one of our engineers work through with you to find the root cause.

    Best regards,

    Brian.

  49. Jianwei says:

    Hi, Smith,

    This is a great web. There are several "Failed But Not Blocking Correlation" events in my queue and seems that I am not able to cancel them.

    I am wondering whether should I delete these events in the queue? if should, how can I do?

    Thanks

  50. jjin says:

    Hi, Brian,

    This is a great web. I am not having several ‘Failed But Not Blocking Correlation’ in the queue for days, and I am not able to cancel them.

    I am wondering should I delete them or I may just leave them there? if these events should be cancelled, how can I do it?

    Thanks,

    JJw

  51. BriSmith says:

    Hi JJw,

    As these have already failed then there is nothing to cancel.  They will get cleaned up based on the settings in the Queue Settings page – so no need to do anything.  In some cases these jobs cab be re-tried, but without knowing what jobs these are I do not know if this makes sense for you.

    Best regards,

    Brian.

  52. SteveW says:

    Hi Brian,

    This is a duplicate post but relevant to stuck Queues.

    If you could answer this you would make a whole bunch of very frustrated people very very happy…..  ;-)

    (1) I take a full farm backup using STSADM of my prod box which has PS2007 running on WSS 3.0

    (2) Then I attempt to create a copy of the farm on a different server ( the DEV server ). I create an installation of PS2007 including up to PWA.

    (3) I delete all databases except the sharepoint management dbs.

    (4) I create a new SSP called SSP2, and move the management and PWA to it. It has a new database name. I delete the original SSP so that I can restore onto the box.

    (5) I then restore the prod farm from backup using:

    STSADM -o restore <\DEV_SERVERShare> <Prod_GUID_from_the prod_farm_backup> -restoremethod new

    (6) Then I make sure I move the PWA and admin into the new SSP and delete the old SSP2, but not delting the SSP2 database ( attempting to delete the db causes it to fail ).

    (7) I do an IISRESET.

    (8) I then run RelinkAllWSSSites http://<dev_server&gt; http://dev_server>/PWA ( note if I use the port ID seems to make it fail, so I ignore the port ID ). Thsi links all the original PWA project sites intot he new system.

    Now – it all works BUT the QUEUE seems to ignore the new system.

    Brian, please do you have any ideas on why the queue might do this? I tried delaying the start to let SQL catch up but no joy.

    Is there a better way to duplicate a farm and have a different server name on the destination server? In effect I am migrating prod to dev.

    Many thanks – a good solution will solve I think a whole lot of pain for a lot of people.

    Thanks in advance,

    SteveW

  53. BriSmith says:

    Duplicate response to that posted to other thread…

    Hi SteveW,

    Not sure where you got the steps from but I haven’t seen this process used before and can imagine that it leaves some disconnection in the relationship between the PWA site and the project application.  If you just install your dev box to get Central Admin working then do a full farm restore – change the name of the server – all should be well.  See my postings on moving production to development http://blogs.msdn.com/brismith/archive/2008/09/26/project-server-2007-moving-a-copy-of-production-to-test-part-2.aspx and http://blogs.msdn.com/brismith/archive/2008/09/20/project-server-2007-moving-a-copy-of-production-to-test-part-1.aspx.

    I don’t know of a supported way to get you working the way you have migrated.

    Best regards,

    Brian

  54. SteveW says:

    Hi Brian,

    I did as you suggested and it worked – but – the Issues and Risks for each project in its site i.e. /PWA/<project_name> seem to be broken, but Project Change Requests is fine.

    Do you any suggestions please? All the rest of the functionality including ebable able to  open projects , save & publish from a client PC using PP2007 to this new server seems fine.

    I did try to use the RelinkAllWSSSites.exe to try and fix the issue, but it keeps giving me attitude , saying :

    "ERROR: Unable to find server http://<server_restored_to&gt; . Check the imput parameters and try again."

    Now I know PWA and the individual project sites are there, so do you have any suggestions please? Is there a need to alter the RelinkAllWssSites.exe.config  file to have the new server name in it?

    I tried this and got the same error.

    Also, is it necessary to run the relinkallwsssites at all?

    Is there a tool to check and correct any broken linkages within the whole system?

    Solving this problem will enable us to push forward in using project server and its frustratingly close to being robust.

    Any thoughts welcome…….

    Cheers,

    SteveW.

  55. BriSmith says:

    Hi Steve,

    I haven’t seen problems with full farm backup – and you shouldn’t need to run the WSS relink tool in this scenario.  You could try synchronizing permissions on the Project Workspace page.  Does you dev environment recognize the same user credentials (domain) as the original?

    Best regards,

    Brian.

  56. SteveW says:

    Hi Brian,

    Yes the dev environment will recognise the same user credentials.

    I havent tried synchronising permissions – could you suggest the best way to do it please , or is it self explanitory?

    The error I get is “page not found” when I try looking at the Issues and Risks by the way. In cruising some of the other news groups, losing Issues and Risks and documents seems to be a recurring theme, if that helps at all.

    Thanks

    Cheers

    SteveW

  57. SteveW says:

    Updated 10/13/2008

    ————————–

    Hi Brian,

    I tried to do syncronization using this method:

    PWAServer SettingsOperational PoliciesProject workspaces

    I then highlighted a random project by clicking on the line for the project. While highlighted i went up to the "synchronise" button up the top of the page & clicked on it.

    I left it run for 5 minutes, tried to look at an issue or risk, and got the exact same error as before "file not found".

    What would you suggest from here please? On the site page for this project, Ican see all the issues and risk listed, but when I click on any isue or risk I get  the "page not found" error.

    I checked the queue and it says "Failed but not blocking correlation" which may be a whole new issue……..

    By the way, I was perfoming these tests as a  Project Sevrer administrator.

    Any suggestions most welcome..    :-)

    Cheers

    Steve.

  58. KH says:

    Hi Brian,

    I’ve sent you a comment a couple of days ago about the constant event logs I’m getting, i.e. Errors 7758, 7761 and 7754 related to the project queue and still frustratingly having no clue to the problem. Do you have any idea how these errors come about?

    Any hints most appreciated.

    Thanks,

    KH

  59. BriSmith says:

    Hi KH,

    Sorry – the earlier comment did not appear to get to me.  I checked my server and also found some of these IDs, and without the description I cannot be certain they are caused by the same issue for you – but it appears my SQL Server went off line for a short time early yesterday morning – and these messages all relate to lost connections to the server from the queue process.  Are you seeing these during normal production hours?  Could you have had some temporary outage?

    Best regards,

    Brian.

  60. KH says:

    Hi Brian,

    You have given me a good hint, though our problem is not exactly the same. Your comment have led us to relook at the SQL Server and we finally found the cause of the problem.

    Although the SQL Server is installed on the same machine as the Project Server, the DB files are located physically on a SAN disk. It is the SAN disk that is giving the problem as it is taking a long time in every transaction – we verifed it by shifting the DB files to the local drive and the problem was resolved.

    Thanks you. You have been very helpful.

    KH

  61. MAV says:

    Hi Brian,

    We have a similar problem as KH however, our database files are on a local drive. The queue is stuck on one job "Reporting (Resource Sync)" with a job state of "Processing". I’m getting the same errors 7758, 7761 and 7754. Checked the ULS logs with no luck.

    Our SQL server is a VM and our Project Server is on a different VM.

    Is there a way to forcefully cancel this job from the database?

    Any help is greatly appreciated!

    Thanks,

    MAV

  62. Una breve y concisa explicación sobre los estados de la cola y como enfrentar algunos problemas cotidianos

  63. smckeown says:

    Hello,

    I am currently having the opposite problem. My queue is running away.  The server load is running around 80% CPU and memory usage.  When I try to select "Manage Queue" the request times out.  

    I dialed the queue setting back to one thread and set all the polling and retries to the maximum time to try to give resource back to the server.  

    Even so, this has been going on for two days now.  At first I thought I should let it run and it will eventually clear itself up.  But that does not seem to be the case.

    Any help is welcome!

    smckeown

  64. Ruse says:

    Hello,

    Could you please help me with the error I get when I try to publish my project:

    <?xml version=”1.0″ encoding=”utf-16″?> <errinfo>

      <general>

        <class name=”Reporting message processor failed”>

          <error id=”24006″ name=”ReportingProjectChangeMessageFailed” uid=”1c8dc596-cbd0-46e8-a4a8-313c8e8057d7″ QueueMessageBody=”Project UID=’3e148e2d-f56c-4ff8-8fbb-9ad54a4c6c72′. PublishType=’ProjectPublish'” Error=”Violation of PRIMARY KEY constraint ‘PK_MSP_EpmTask’. Cannot  insert duplicate key in object ‘dbo.MSP_EpmTask’.&#xD;&#xA;The  statement has been terminated.” />

          <error id=”24006″ name=”ReportingProjectChangeMessageFailed” uid=”6d6b1619-8726-476d-9a8f-8c374d23f966″ QueueMessageBody=”Project UID=’3e148e2d-f56c-4ff8-8fbb-9ad54a4c6c72′. PublishType=’ProjectPublish'” Error=”The INSERT statement conflicted with the FOREIGN KEY constraint  &quot;FK_MSP_EpmAssignment_ProjectUID_TaskUID&quot;. The conflict  occurred in database &quot;ProjectServer_Reporting&quot;, table  &quot;dbo.MSP_EpmTask&quot;.&#xD;&#xA;The statement has been terminated.” />

          <error id=”24006″ name=”ReportingProjectChangeMessageFailed” uid=”064cf886-db55-4e94-befa-6521a4908ac7″ QueueMessageBody=”Project UID=’3e148e2d-f56c-4ff8-8fbb-9ad54a4c6c72′. PublishType=’ProjectPublish'” Error=”The INSERT statement conflicted with the FOREIGN KEY constraint  &quot;FK_MSP_EpmAssignment_ProjectUID_TaskUID&quot;. The conflict  occurred in database &quot;ProjectServer_Reporting&quot;, table  &quot;dbo.MSP_EpmTask&quot;.&#xD;&#xA;The statement has been terminated.” />

          <error id=”24006″ name=”ReportingProjectChangeMessageFailed” uid=”07c2c2b0-6077-4a75-85b8-78b5d339b4a0″ QueueMessageBody=”Project UID=’3e148e2d-f56c-4ff8-8fbb-9ad54a4c6c72′. PublishType=’ProjectPublish'” Error=”The INSERT statement conflicted with the FOREIGN KEY constraint  &quot;FK_MSP_EpmAssignment_ProjectUID_TaskUID&quot;. The conflict  occurred in database &quot;ProjectServer_Reporting&quot;, table  &quot;dbo.MSP_EpmTask&quot;.&#xD;&#xA;The statement has been terminated.” />

          <error id=”24006″ name=”ReportingProjectChangeMessageFailed” uid=”d10e72bd-2c32-4250-a1a2-c3babca56c02″ QueueMessageBody=”Project UID=’3e148e2d-f56c-4ff8-8fbb-9ad54a4c6c72′. PublishType=’ProjectPublish'” Error=”The INSERT statement conflicted with the FOREIGN KEY constraint  &quot;FK_MSP_EpmAssignment_ProjectUID_TaskUID&quot;. The conflict  occurred in database &quot;ProjectServer_Reporting&quot;, table  &quot;dbo.MSP_EpmTask&quot;.&#xD;&#xA;The statement has been terminated.” />

          <error id=”24006″ name=”ReportingProjectChangeMessageFailed” uid=”f69cdfe6-c5cf-4cdc-8af6-f8018317d958″ QueueMessageBody=”Project UID=’3e148e2d-f56c-4ff8-8fbb-9ad54a4c6c72′. PublishType=’ProjectPublish'” Error=”The INSERT statement conflicted with the FOREIGN KEY constraint  &quot;FK_MSP_EpmAssignment_ProjectUID_TaskUID&quot;. The conflict  occurred in database &quot;ProjectServer_Reporting&quot;, table  &quot;dbo.MSP_EpmTask&quot;.&#xD;&#xA;The statement has been terminated.” />

        </class>

        <class name=”Queue”>

          <error id=”26000″ name=”GeneralQueueJobFailed”  uid=”f8f68b30-8124-41ae-9ab2-bc96e50a455e” JobUID=”205f41d4-18ba-4a83-859d-0826ad566b44″ ComputerName=”PROJSRV” GroupType=”ReportingProjectPublish” MessageType=”ReportProjectPublishMessageEx” MessageId=”1″ Stage=”” />

        </class>

      </general>

    </errinfo>

    What could be the cause?

    Thank you.

    Ruse

  65. Karen says:

    Have the problem with my queue being stuck, and was told by a project expert that the order in which the services are starting is incorrect.  He said there was a disconnect between project and sql and there were permission problems, Restarting the server and starting the services in the right order worked.  Well the system admin did an upgrade and rebooted the server now everything is stuck again.  The project expert did not tell me the order that the services had to start.  Do you know what order the services have to start in order to not have a queue problem?

    Thanks

    Karen

  66. BriSmith says:

    Hi Karen,

    I believe there was an issue that if SQL Server was restarted then the Project Services would get left hanging – but this was fixed some time ago in a service pack or CU.  If you are still suffering from something that looks like this then re-start the Project queue and event services. But it could be something else…

    Best regards,

    Brian.

  67. Ashu says:

    Plz provide the resulation if phase same kind of problem

  68. BriSmith says:

    Hi Ashu,

    There isn’t a single resolution – and it is not clear what problem you are facing.

    Best regards,

    Brian

  69. Andrew says:

    We are experiencing the same issue with the queue being stuck. The status is currently "Waiting to be Processed" and it inumber 1 on the list. I have checked previous jobs back 12 months and nothing is still hanging there. I have re-started the services twice and rebooted the machine, still no go.

    I am trying to save a new project to the system for the first time.

    Also, which logs should I bee looking at

    Any ideas.

  70. BriSmith says:

    Hi Andrew,

    Sounds like the queue service isn’t running.  Check using task manager – you should see two instances of the queue service and two of the event service.  If you see none then the service is probably stopped – if you see just one then for some reason it hasn’t spawned the 2nd instance which processes the queue jobs.  Restarting the service would be the best first step – and if things are still bad then the logs created when the queue tried to restart should give a good idea what went wrong.

    Best regards,

    Brian.

  71. Luba says:

    We just restored 4 databases from prod to staging and then re-provisioned the site. This kicked off some 700 "Reporting (Resource Sync)" jobs and "User Synchronization" job. Looks like last one has been processed while others are still in "Waiting to be processed" mode. We have ~6000 resources and ~1500 projects in prod.

    We started getting strange errors in Event Viewer right away. Since then I re-provisioned the site again, restarted the server, restarted both services on the server, gave admin account dbowner rights on all 4 databases. Right now it keeps logging errors in Event Viewer on the server every minute and all jobs in the queue are at 0% completion. Nothing is being processed. Any idea on what could I be missing?

    Thanks,

    Luba

  72. Luba says:

    Addition to previous message- we are getting the following 3 types of errors repeatedly:

    Standard Information:PSI Entry Point:

    Project User: xxxx

    Correlation Id: 1cf8978f-bd5b-4b50-92c9-1c64904ccff6

    PWA Site URL: http://staging.domain.com/ProjectServer

    SSP Name: PRISM Stage SSP

    PSError: Success (0)

    Queue System Restarting due to an unexpected error. Queue type (Project, Timesheet queue etc): ProjectQ.  Error: System.Xml.XmlException: Root element is missing.

      at System.Xml.XmlTextReaderImpl.Throw(Exception e)

      at System.Xml.XmlTextReaderImpl.ParseDocumentContent()

      at System.Xml.XmlLoader.Load(XmlDocument doc, XmlReader reader, Boolean preserveWhitespace)

      at System.Xml.XmlDocument.Load(XmlReader reader)

      at System.Xml.XmlDocument.LoadXml(String xml)

      at Microsoft.Office.Project.Server.Errors.PSError..ctor(String xmlString)

      at Microsoft.Office.Project.Server.BusinessLayer.Queue.Receiver.CleanupCompletedThreads()

      at Microsoft.Office.Project.Server.BusinessLayer.Queue.Receiver.ThreadEntry()

    ===========================

    Standard Information:PSI Entry Point:

    Project User: xxxx

    Correlation Id: 1cf8978f-bd5b-4b50-92c9-1c64904ccff6

    PWA Site URL: http://staging.domain.com/ProjectServer

    SSP Name: PRISM Stage SSP

    PSError: Success (0)

    An unxpected exception occurred in the Project Server Queue. Queue type (Project Queue/Timesheet Queue): ProjectQ. Exception details: Root element is missing..

    ======================

    Standard Information:PSI Entry Point:

    Project User: xxxxx

    Correlation Id: 1cf8978f-bd5b-4b50-92c9-1c64904ccff6

    PWA Site URL: http://staging.domain.com/ProjectServer

    SSP Name: PRISM Stage SSP

    PSError: Success (0)

    An unxpected exception occurred in the Project Server Queue. Queue type (Project Queue/Timesheet Queue): ProjectQ. Exception details: The site with the id f36ddbb6-5927-4c2b-8c32-d484681f1575 could not be found..

    Luba

  73. BriSmith says:

    Not sure what happened here Luba.  The jobs kicked off after a provision may take a while – or could cause a deadlock.  Did you re-restore the databases after the first provision?  The bad XML error looks like something is not good in the queue tables.  Might need a support incident for one of my colleagues to help you out.

    Best regards,

    Brian.

  74. HyderZaidi says:

    Hi Brain,

     I am getting very strange behaviour on one of my Master Project. When-ever, I published the job it throws below exception. Also, another strange behaviour about this project is, it automatically stops generating a notification/alert message (particularly issue with this project only out of 200+ projects). Your expert adivse is needed to grap the problem and move towards the resolution stage :)

    Your ReportingProjectPublish job failed.  Its current state is FailedNotBlocking.  It was 0% complete.  It entered the queue at 02/13/2010 15:09:04.

    To get more information about the job failure, please go to Project Web Access.  Select Personal Settings from the left menu.  Then select My Queued Jobs.

    The errors returned from the queue are as follows:

    Error ID: 24006

    Error ID: 26000

    Detailed error below – send it to the administrator for more detailed troubleshooting.

    <?xml version="1.0" encoding="utf-16"?>

    <errinfo>

     <general>

       <class name="Reporting message processor failed">

         <error id="24006" name="ReportingProjectChangeMessageFailed" uid="30e983c1-4e90-493e-a2bb-605bd86db52c" QueueMessageBody="Project UID=’2f29f726-9cd7-40d8-96c3-370e4ae05004′. PublishType=’ProjectPublish’" Error="Exception of type ‘System.OutOfMemoryException’ was thrown." />

         <error id="24006" name="ReportingProjectChangeMessageFailed" uid="fe94bad8-2e64-418e-848d-0bd846ec0788" QueueMessageBody="Project UID=’2f29f726-9cd7-40d8-96c3-370e4ae05004′. PublishType=’ProjectPublish’" Error="Exception of type ‘System.OutOfMemoryException’ was thrown." />

         <error id="24006" name="ReportingProjectChangeMessageFailed" uid="b8b36074-c231-42e9-9102-0caa7a890d51" QueueMessageBody="Project UID=’2f29f726-9cd7-40d8-96c3-370e4ae05004′. PublishType=’ProjectPublish’" Error="Exception of type ‘System.OutOfMemoryException’ was thrown." />

         <error id="24006" name="ReportingProjectChangeMessageFailed" uid="08508e9e-4c34-4aeb-b998-8fddf6d16368" QueueMessageBody="Project UID=’2f29f726-9cd7-40d8-96c3-370e4ae05004′. PublishType=’ProjectPublish’" Error="Exception of type ‘System.OutOfMemoryException’ was thrown." />

         <error id="24006" name="ReportingProjectChangeMessageFailed" uid="f361ac92-35d6-4f80-afbd-72b34bdc03af" QueueMessageBody="Project UID=’2f29f726-9cd7-40d8-96c3-370e4ae05004′. PublishType=’ProjectPublish’" Error="Exception of type ‘System.OutOfMemoryException’ was thrown." />

         <error id="24006" name="ReportingProjectChangeMessageFailed" uid="ab2bdad0-420f-4c27-9505-ebe1b03d2748" QueueMessageBody="Project UID=’2f29f726-9cd7-40d8-96c3-370e4ae05004′. PublishType=’ProjectPublish’" Error="Exception of type ‘System.OutOfMemoryException’ was thrown." />

       </class>

       <class name="Queue">

         <error id="26000" name="GeneralQueueJobFailed" uid="c8b5407b-ce9c-4303-9a5d-c14594ca948c" JobUID="810177be-e177-4911-bec1-3c6e33c6f328" ComputerName="EPMAPP" GroupType="ReportingProjectPublish" MessageType="ReportProjectPublishMessageEx" MessageId="1" Stage="" />

       </class>

     </general>

    </errinfo>

    Regards,

    Hyder Zaidi

  75. Hi Brian,

    I’ve been trying to resolve the subject error in my Project Server 2007 farm.  Nothing appears to be broken, but I get this warning message every couple of minutes.  The only reference I’ve been able to find on the Web has this error occurring in SharePoint where the solution was to use the command: stsadm -o sync -deleteolddatabases 1 to resolve it.  Unfortunately, this command does not exist in Project Server.

    Any suggestions?

    Thanks!!!

  76. Sorry, forgot the error message…

    Synchronization for Shared Services Provider ‘SharedServices1′ has failed. The operation will be retried.

    Reason: Collection was modified; enumeration operation may not execute.

    Technical Support Details:

    System.InvalidOperationException: Collection was modified; enumeration operation may not execute.

      at System.Collections.Hashtable.HashtableEnumerator.MoveNext()

      at Microsoft.Office.Server.Administration.SharedResourceProvider.SynchronizeConfigurationDatabaseAccess(SharedComponentSecurity security)

      at Microsoft.Office.Server.Administration.SharedResourceProvider.SynchronizeAccessControl(SharedComponentSecurity sharedApplicationSecurity)

      at Microsoft.Office.Server.Administration.SharedResourceProvider.Microsoft.Office.Server.Administration.ISharedComponent.Synchronize()

      at Microsoft.Office.Server.Administration.SharedResourceProviderJob.Execute(Guid targetInstanceId)

    For more information, see Help and Support Center at

  77. BriSmith says:

    Hi Bill,

    What exactly is the warning you are getting?  The stsadm command may still be relevant if it is some reference to a non-existent Project entity in the SharePoint system.

    Best regards,

    Brian.

  78. Luis Crespo says:

    I have one project stuck at 97% complete publishing however the user lost connectivity and now the job is stuck and cannot be canceled and the project cannot be checked in.  How do I cancel this job so I can force check in the project back in.  I am using Project Server 2007 and Project Pro 2007 SP1.

  79. BriSmith says:

    Hi Luis,

    If it is publishing then losing connectivity is not the issue.  only save from Project Professional jobs require connectivity.  If it has stopped in publish then it should be possible to cancel – or wait for completion if it still says publishing – or check that another job for this project isn’t further back in the queue.  You might need a support call to help you through this – but also being on the latest CU will ensure you have any fixes we have released for issues you might hit during publish.

    Best regards,

    Brian.

  80. Chris says:

    I get the same problem today from a user using the save function. It got stuck for whatever reason and now the job is stuck in the queue and user is unable to do anything except read only.

    Any recommendation?

    thanks

    Chris

  81. BriSmith says:

    The queue job should cancel Chris, but depending on the release level of your system you may need to check ‘cancel jobs getting enqueued’ assuming the user cannot re-connect.  Once this is cancelled then you should be able to check in.  As I’ve mentioned in various posting the ‘force’of force check-in is a bit of a misnomer.

    Best regards,

    Brian

  82. gary.mcneeley says:

    Hi Brian,

    Hoping you can help with a Timesheet Queue error I am receiving.  When certain timesheets are sent, a TimesheetLineHasNonExistentItem error occurs and block the queue.  Here is the error message:

    [ApplicationException: Queue request failed "Failed" Job ID: 7ccf575b-4245-463b-ac5b-d339f103b5a6.

    <?xml version="1.0" encoding="utf-16"?>

    <errinfo>

     <general>

       <class name="ASSN_UID">

         <error id="3213" name="TimesheetLineHasNonExistentItem" uid="d42a2291-cc0e-4cb3-b200-4dd6c7d3f74b" columnName="ASSN_UID" value="6cd3edac-43b7-4938-84b6-8e0358e9f8d8" />

       </class>

       <class name="Queue">

         <error id="26000" name="GeneralQueueJobFailed" uid="f7bacf3a-c48a-4b9c-9896-5140ca79da65" JobUID="7ccf575b-4245-463b-ac5b-d339f103b5a6" ComputerName="AMEDWSAPGR20" GroupType="TimesheetUpdate" MessageType="UpdateTimesheetMessage" MessageId="1" Stage="" />

       </class>

     </general>

    </errinfo>]

      ChppmPSI.QueueSystem.WaitForQueue(Guid jobId) in C:projectsappsOPMDevOPM WebChppmPSIQueueSystem.cs:155

      ChppmPSI.Timesheet.CompleteTimesheet(TimesheetDataSet dsTS, Guid tsPeriod, Int32& jobStatus, Boolean WaitforQueue) in C:projectsappsOPMDevOPM WebChppmPSITimesheet.cs:417

    [Exception: There was an error completing the timesheet.  Please try again later.  Error: Queue request failed "Failed" Job ID: 7ccf575b-4245-463b-ac5b-d339f103b5a6.

    <?xml version="1.0" encoding="utf-16"?>

    <errinfo>

     <general>

       <class name="ASSN_UID">

         <error id="3213" name="TimesheetLineHasNonExistentItem" uid="d42a2291-cc0e-4cb3-b200-4dd6c7d3f74b" columnName="ASSN_UID" value="6cd3edac-43b7-4938-84b6-8e0358e9f8d8" />

       </class>

       <class name="Queue">

         <error id="26000" name="GeneralQueueJobFailed" uid="f7bacf3a-c48a-4b9c-9896-5140ca79da65" JobUID="7ccf575b-4245-463b-ac5b-d339f103b5a6" ComputerName="AMEDWSAPGR20" GroupType="TimesheetUpdate" MessageType="UpdateTimesheetMessage" MessageId="1" Stage="" />

       </class>

     </general>

    </errinfo>]

    Any help appreciated.  Thanks!

    Regards,

    Gary McNeeley

  83. Muhammad Ehsan says:

    Hi Brian,

    Excellent column. I am facing an issue which has now consumed a lot of our time. When I am saving projects from project professional it crashes!! I am not sure what is going on. When I go and check the queue it says that "Getting queued" which is because project professional crashed without completing the save job. Projects which are already saved have no issues at all.

    Also if I am saving using the main admin account (using which I did the installation) everything works fine. It was not happening before so I am not sure what is causing this to happen. We are right in the middle of a rollout and this is causing us to delay everything. Your help will be highly appreciated.

    Thanks!!

  84. BriSmith says:

    Hi Muhammad,

    Are you on the latest cumulative updates? I'd suggest using a different profile/account to connect, as it may somehow be that the current one is corrupt (this is supported by the fact it works for the admin but not your other account).  Another option would be to clear the cache, but just adding a new Project Server account in Pro (File, Info, Project Server Accounts) will give a completely new cache.. just use  a new name but the same URL.  Let me know how this goes.

    Best regards,

    Brian.

  85. maria says:

    Hi Brian,

    Do you know if exist some powershell commands in order to stop for some minutes and then start the Microsoft project Queuing Service?

    Thanks a lot!

  86. xavier says:

    Hi Brian Smith,

    I like your news !

    I had a problem with my queue service (lot of tasks in "Waiting to be processed …")

    With your link : technet2.microsoft.com/…/348d1f05-5cc6-41bb-be2c-ea28bbf1c3421033.mspx

    And commands to restart the service :

    net stop ProjectQueueService

    net start ProjectQueueService

    it WORK now o/

    Thx a lot :)

  87. Hammad says:

    When I restart the SQL Server Services, The queue sometimes stucks and do not work until I restart the application server. Can anyone inform me any better way to deal with it in which I may not have to restart the Application server?? Thanks!!

  88. BriSmith says:

    Hi Hammad, I certainly wouldn't restart SQL Server without first stopping SharePoint and the Project services anyway – that is a very dangerous practice. Why are you re-starting SQL?  But if it does happen then re-starting the application service would be the right thing to do.

    Best regards,

    Brian.

  89. Hammad says:

    Hi Brian,

    I am using MSPS 2007 with SQL Server 2005.

    We had deleted many projects using a script but when we looked into the queue of Project Server we found a long queue. It appeared that the delete job was taking a long time to delete the projects. So as a result, we canceled the deletion of projects from the queue. Now the queue is finished. But still I see many delete requests on the SQL server with one waiting for the other. There is no deadlock. But puts Lonk in the reporting database.

    Each one is calling following SP:

    XXXProjectServer_Reporting.dbo.MSP_Epm_DeleteProject;1

    My questions:

    Why it is taking ~10 minutes to delete a project?

    If I have canceled the delete job from the Project Server queue and the job still appear in the Activity monitor of SQL server then can I kill that job from SQL server?

    Can you suggest and change in configuration anywhere which help me to improve the speed?

    Thanks!

  90. BriSmith says:

    Hi Hammad – You don't mention what script you used to delete projects – but I assume it just issued PSI calls to handle the deletions in a supported fashion.  The jobs will go on to the queue and the whole idea of the queue is to control the feeding of jobs through the application tier and SQL Server – so I am not sure why you felt the need to delete the remaining jobs from the queue.  If the queue jobs have registered as completed then I would not expect activity to still really be active – although activity monitor may show some recent jobs.  I cannot say if 10 minutes is normal or not for project deletion as you give no indication of size or complexity of the projects or the power of your SQL Server.  If it is putting a lock on your reporting database is this really what the issue is? I'd suggest opening a support incident to have my colleagues help you understand what is going on.

    Best regards,

    Brian.

  91. Ankit Jain says:

    The timesheet of the resource did not reflect in the work plan. What can cause such issues? This is what I suggested –

    Can you cancel the jobs running for the affected work plan? Then, please try to republish the affected work-plans:

    Open the offended plan in MS Project Client as usual

    Choose File > Save for Sharing; Do not change the plan name. The operation saves the plan locally.

    Choose File > Save As. Click "Save" to save the plan back to Project Server

    Notice the plan name is already populated and non-editable.

    Verify the save operation is finished by checking the "Active Cache Status"

    When the save is done, Save the plan, wait the save is completed and Publish it as usual.

    However when he tries to cancel the jobs in the manage queue page he sees an error – Root element is missing.

    HOWEVER; THIS ARTICLE IS AMAZING.

  92. Project Server DBA says:

    Hi,

    During last one week, many of the jobs have started failing. The job usually process successfully but sometimes fails randomly with state ‘Failed But Not Blocking Correlation’.

    Whenever the job fails, it exactly fails at following percentage:

    OLAP Cube Build  –> 50%

    Project Checkin  –> 12%

    Project Publish  –> 96%

    Project Publish Notifications  –> 0%

    WSS Workspace Create  –> 0%

    Can you please inform me that what can be the reason of failing these jobs?

    Thanks!!

  93. Chris OK says:

    Hi Brian,

    Our PS2007 environment has a job (AD Sync) that is not completing and it taking up 100% of the CPU (Queue service & SQL Server).  

    This job has been running for over two days now.  We have tried rebooting the server as well as cancelling the job but it is not fixing the issue.  After a rebootm the CPU returns to 100% once the job is picked up again.

    This environment has been running for many years and we have not ezperienced this issue before.

    Is there any way to remove this job from the Queue that is not completing?

    Any suggestions on how to resolve this issue?

    Thanks,

    Chris

  94. Chris OK says:

    We are running PS 2007 v12.0.6422

    Thanks,

    Chris

  95. Hello,

    Sorry didn't know how to start out a new Comment.

    Can someone tell me where all i can look for MS Project 2007 Log files.    

    We have custom apps with SAP pulling data from project.

    PWA > Applied Task and Errors shows – Shoes alot of errors that was showing missing data from a few days ago.

    1- First question is how do you re-process this type of file/task that was not published.

    2- Second where can i find the different Log Files?

    I have looked at the following places:

    A- C:Program FilesCommon FilesMicrosoft SharedWeb Server Extensions14Logs

    B- Event Viewer > Windows Logs > (Applicaton)-&-( System ) Logs

    I look at/inside C:Program FilesCommon FilesMicrosoft SharedWeb Server Extensions14Logs

    but I dont' see anything todo with MS Project – for ULS Logs either.

    Please help!

    Just want to know where all someone can look for log files.

  96. BriSmith says:

    Hi MSPS1,

    The forums social.technet.microsoft.com/…/project or a support incident might be better for questions that don't relate to a blog post.  You have the right places for the log files.  USL logs (A) would be the most likely place to find problems – but until I know the errors I can't give much guidance on how to resubmit.  I'd ask the people that set up the interface to SAP.

    Best regards,

    Brian.

  97. Veeru says:

    Hi,

    Archive Global Project,Archive Custom fields,Archive System setting, Archive Security category timesheet queue jobs are getting queued and the job status is in Waiting to be precessed.

    Could any one please let us know why this is getting stuck in the queue, its happening daily.

    However is tried cancelling the job, however its getting stuck everyday.

    Please let us know the solution to this issue.

    Thanks in advance

    Veeru

  98. BriSmith says:

    Hi Veeru, I'd suggest you review the queue history – if the queue is generally working then there may be an old failed job blocking this one.

    Best regards,

    Brian.

  99. Hasan Jamal Siddiqui says:

    Hi,

    Nice blog :) .

    I ma facing issue freqentaly in all project as "Your queue job UpdateProjectSitePath failed. Please contact your administrator for assistance" and error code is  WSSWebDoesNotExist

    when try to give permsiion to any body on project.

    Kindly provide appropriate solution.

  100. BriSmith says:

    When you update users Hasan, then it will sync the user to the project site.  It sounds here like the site no longer exists – but is still referenced in the page showing Project Sites under Server Settings in PWA.  Removing the site from that list may solve this issue.

    Best regards,

    Brian

  101. mahi_SP says:

    Hi Brian,

    This happens in my environment every 25 days or so…Any hint for solving this…

    TimeCreated      : 4/8/2014 5:53:50 AM

    Id               : 7754

    LevelDisplayName : Critical

    Message          : Standard Information:PSI Entry Point:

                      Project User: xxxxxxxxxxxxxxx

                      Correlation Id: 4bfa3fad-b1f4-4810-8535-0de6a21207bd

                      PWA Site URL: http://msproj2010/PWA

                      SSP Name: Project Server Service Application

                      PSError: NoError (0)

                      Queue unable to interact with SQL. Queue type (Project Queue

                      , Timesheet Queue etc): ProjectQ Exception: Microsoft.Office

                      .Project.Server.BusinessLayer.Queue.QueueSqlException: LockN

                      extAvailableGroup failed —> System.Data.SqlClient.SqlExcep

                      tion: An invalid application lock resource was passed to xp_

                      userlock.

                         at System.Data.SqlClient.SqlConnection.OnError(SqlExcepti

                      on exception, Boolean breakConnection)

                         at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarni

                      ng(TdsParserStateObject stateObj)

                         at System.Data.SqlClient.TdsParser.Run(RunBehavior runBeh

                      avior, SqlCommand cmdHandler, SqlDataReader dataStream, Bulk

                      CopySimpleResultSet bulkCopyHandler, TdsParserStateObject st

                      ateObj)

  102. Chris Karanski says:

    Your article references a link to a TechNet article on the Queue, but that link is a dead end.   Might you have an updated link to that article??

    Thanks!!!

    Chris

    First I will point to a great TechNet article on the Queue and as you will all have read this then my explanations will make more sense :).

  103. BriSmith says:

    Sorry for the broken link Chris – pretty sure this is the same article I was referencing – technet.microsoft.com/…/cc197395(v=office.12).aspx – I'll update the post too.

    Best regards,

    Brian.

  104. Darko says:

    Hi, a newbie question:

    What you do when you have a several months old jobs in "Waiting to be processed " status? You cancel them?

    Thanks

  105. BriSmith says:

    Hi Darko – depends on the jobs – the best way would be to get them unblocked (cancel any failed and blocking job) – but you could cancel.  Be aware that some cancels will lost data – such as a save from Project Professional – whereas cancelling a publish is no big deal – you just publish again.

    Best regards,

    Brian.