Urgent - where in database schema is 'IN PROGRESS'

[ftoth]

Hi again,

I have just had a 4th deployment wedge the system in "IN PROGRESS".
See this for the earlier part of the story:

http://forums.alfresco.com/en/viewtopic.php?f=29&t=14763

The only way out of this has been to rebuild from backups, but this too has
its problems, as I am getting subtle inconsistencies from my backups.

I'm hoping someone from Alfresco can give me a hint of how to patch the database
on these deployments. One of the oddities is that the deployments succeed perfectly.
The problem is that Alfresco doesn't correctly update the status from "IN PROGRESS"
to "LIVE" or whatever.

Since my Alfresco instances are rapidly becoming useless, I'm hoping I can
patch the broken deployment and move on, instead of having to roll back
each time this happens.

Again, for emphasis: These are production instances with hundreds of successful
deployments. Now, in the last few days, this problem has shown up on two
different unrelated systems!

Any help appreciated.

Thanks,

Fred

[mrogers]

Clearly there is something strange going on here.   I've been following your postings.

I don't think IN PROGRESS is stored in the database.   I'm certain of this for versions 2.2 upwards.

[ftoth]

Hi,

Thanks for this. Yes, I'm afraid I might be the only user out here trying
to do real production with 2.1 Community. I know that there's been a lot
of reworking of the deployment mechanism in the later versions, so it's
easy to imagine that this bug is limited to 2.1.

Thanks,

Fred

[ftoth]

I'm now trying to decide which of these options is the most likely to
get me out of the woods:

1. Start with a fresh install of 2.1 community. I give up all my configuration, document
history, etc., but this might weed out the instability.

2. Migrate to labs 3b. Untested by me, but might be more stable. Might not be.

3. Migrate to HEAD. As above.

Enterprise really isn't an option for us. We're building sites dedicated to tropical diseases and
it's a requirement that we be able to replicate these systems in the disease endemic countries.
These systems must be 100% open source with no fees. Thus we are another victim of
Alfresco's change in release policy.

Thanks,

Fred

[mrogers]

3B is the most solid community WCM version, if you choose to upgrade use that.   But let's not go there yet. 

Something has happened to your environment in the last few days.    I'm fairly sure its not your Alfresco environment since that has been stable,  but what it is I don't know.   That's why I was watching your posts over the last few days to give me a clue.  I suspect something with your network, otherwise what else connects your multiple failures?     I need to gen up on 2.1C before I can give more solid info.

[ftoth]

I'm still wondering about Firefox 3.03. This came out 3 weeks ago today. In all 4
cases of this problem, the user was running Firefox 3.03. This is three different
users.

It's unlikely that it's network. One server is in Texas, US, but the primary user is in London, UK.
The other server and primary user are both in São Paulo, Brazil. The third user is me,
supporting both of these systems.

Just to spin this fairy tale a bit further: It could be something to do with the AJAX
part of the deploy monitor. As mentioned previously, the deploys actually work correctly,
but something goes wrong at the end.

I've had a look at some of the code. Once the deploy is stuck in "IN PROGRESS", all the
other tools ("view deployment", deploy snapshot, etc.) fail immediately with NullPointerError.
It looks like a simple matter of the code trying to iterate over snapshots and trying
to load the "finished date". And there's not one there, so we get NPE. I've considered
trying to patch up that bit of code to make it more robust, just to see where that takes me.

Thanks again,

Fred

[ftoth]

It looks like the core problem is an "incomplete" deployment report.

We're deploying to two servers (staging and live). In a normal deployment,
you can browse with the node browser and find two "deployment reports"
which are children of the web site. These deployment reports have things
like snapshot version numbers, start times, end times, a boolean flag for success, etc.
There's also a content reference that gives you the text of the deployment report.

In my case of wedged "IN PROCESS" deployments, one of the deployment reports
is incomplete. It's missing all of that useful stuff, except for the content. There's
a bogus name, no start/end times, etc. Somehow, even though the deployment
succeeded, this deployment report is broken. It even has an aspect of "incomplete".

That's it for now. Tomorrow, I figure out how to get rid of the damn thing!
It appears that if I can do that, the deployment service will simply see that
one of the servers needs deploying to. This should allow us to redeploy to that
server.

Is there a quick and easy way to delete a node? I can easily find the node reference
for this bogus report. What's the quick and easy way to get rid of it?

Thanks,

Fred

[ftoth]

Hi again,

I finally have a simple work-around for this.

After enough code reading and node browsing, I was able to write a script
to delete the broken deployment report. Once this is done, the snapshot
can be redeployed without error.

I still don't know what's causing this problem in the first place, but this
fix means that it's a much smaller problem, since no rebuilding or
even restarting is needed.

Thanks,

Fred

[pconnors]

Hi Fred!  I know this happened back in 2008, but does the script you speak of still exist?  I'm looking to do this exact thing right now.

[ftoth]

Hi,

Yes, the script still exists and we still use it.

You need to put the name of your web project in the script. Store the script in "Data Dictionary/Scripts/fix-bad-deploy.js"
and you can run it with URLs like this:

http://yourhost.com/alfresco/command/script/execute?scriptPath=/Company%20Home/Data%20Dictionary/Scr...
http://yourhost.com/alfresco/command/script/execute?scriptPath=/Company%20Home/Data%20Dictionary/Scr...
http://yourhost.com/alfresco/command/script/execute?scriptPath=/Company%20Home/Data%20Dictionary/Scr...

The first URL will just list your deploy reports. You should see one that's garbled, with null dates, etc.

This is the one that's causing the problem.

The second URL is the same as the first except that includes the contents of the reports. This is the same as the deployment details.

The third will delete the bad report. It's completely harmless to run this even when there's no problem, as it only deletes the garbled report. If there's no garbled report, it doesn't do anything.

I've got the above links set up in a custom template folder view. Makes it simple and easy to run.

Let me know if this works for you.

Fred

logger.log("FIX-BAD-DEPLOY");

var nl = "\n";

var out = "";

var site = companyhome.childByNamePath("Web Projects/YOURWEBPROJECTNAME");
out += "Site: " + site.name + nl;
var siteDeployVersion = site.properties["{http://www.alfresco.org/model/wcmappmodel/1.0}selecteddeployversion"];
out += "Deploy version: " + siteDeployVersion + nl;

// args
var doDelete = false;
if (args["delete"] == "true")
        doDelete = true;
var doContent = false;
if (args["content"] == "true")
        doContent = true;

out += "delete arg: " + doDelete + nl;
out += "content arg: " + doContent + nl;

out += nl;

for each (child in site.children) {
        if (child.type != "{http://www.alfresco.org/model/wcmappmodel/1.0}deploymentreport")
                continue;
        out += "REPORT: " + child.name + nl;

        var deployVersion = child.properties["{http://www.alfresco.org/model/wcmappmodel/1.0}deployversion"];
        out += "deploy version: " + deployVersion + nl;

        var createTime = child.properties.created;
        var deployStart = child.properties["{http://www.alfresco.org/model/wcmappmodel/1.0}deploystarttime"];

        var deployEnd = child.properties["{http://www.alfresco.org/model/wcmappmodel/1.0}deployendtime"];
        out += "created: " + createTime + nl;
        out += "deploy start: " + deployStart + nl;
        out += "deploy end: " + deployEnd + nl;

        out += nl;

        if (doContent)
                out += child.content + nl;

        if (deployVersion == null) {
                if (doDelete) {
                        out += nl + "DELETING: " + child.name + nl;
                        site.removeNode(child);
                        out += nl + "DELETE DONE\n";
                }
        }
}

function result() {
        return("<pre>\n" + out + "</pre>\n");
}
result();
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍