[#DOTCMS-4911] Lucene index has identifiers that don't exist in the database

dotCMS

Lucene index has identifiers that don't exist in the database

Created: July 29, 2010 12:45 PM Updated: August 09, 2010 11:20 AM Due: 7/29/10

Component/s:

None

Affects Version/s:

1.9.1

Fix Version/s:

1.9.1

Time Tracking:

Original Estimate:

Not Specified

Remaining Estimate:

0 minutes

Time Spent:

30 minutes

File Attachments:

None

Image Attachments:

1. web1.png (192 kb)	2. web1_only_shows_4_results_even_though_it_says_it_has_5.png (204 kb)	3. web2.png (181 kb)
4. web2_shows_5_results.png (203 kb)

Description

« Hide

We're running this query to get events from the Query Test Tool
+structureName:event +((event.startDate:[07/29/2010 00:00:00 TO 02/14/2011 00:00:00]) (+(event.startDate:[01/10/2010 00:00:00 TO 07/29/2010 00:00:00]) +(event.endDate:[07/29/2010 23:59:59 TO 02/14/2011 00:00:00]) )) +languageId:1 +deleted:false +live:true +conhost:32af50e2-1c69-45cb-b7da-a826e5dbf0c7*

with Limit: 5 and Sort: event.startDate asc

in web1 I get supposedly 5 results with these identifiers:

INODE : af7515ee-a44d-44f7-ab90-cd63056bb9c9 IDENTIFIER : 401d6e25-25b9-45b5-9d91-cf749425fe2d
INODE : 3eabe7ac-45b4-4a78-b54a-2213f17d71c0 IDENTIFIER : aea6eb4e-086f-4a52-8a77-be90187402d7
INODE : 8f7c3c5a-4e48-47f6-aefe-351121314dc6 IDENTIFIER : 32cfbc11-1785-40f1-a7c7-eff6a9969216
INODE : 8523cfb3-06eb-4f82-a433-71a66a57833c IDENTIFIER : 86fbd746-cb12-41be-99c1-9d796bbb35ac
INODE : c48bde95-f44b-4991-b17e-82360b176139 IDENTIFIER : f5b66171-f0e2-4611-bafc-75b23e02a750

but the list of inodes below only shows 4.

the missing identifier from the list doesn't exist in the database
INODE : c48bde95-f44b-4991-b17e-82360b176139 IDENTIFIER : f5b66171-f0e2-4611-bafc-75b23e02a750

web2 has a different set of results, because it doesn't have the bad identifier in its list
INODE : af7515ee-a44d-44f7-ab90-cd63056bb9c9 IDENTIFIER : 401d6e25-25b9-45b5-9d91-cf749425fe2d
INODE : 3eabe7ac-45b4-4a78-b54a-2213f17d71c0 IDENTIFIER : aea6eb4e-086f-4a52-8a77-be90187402d7
INODE : 8f7c3c5a-4e48-47f6-aefe-351121314dc6 IDENTIFIER : 32cfbc11-1785-40f1-a7c7-eff6a9969216
INODE : 8523cfb3-06eb-4f82-a433-71a66a57833c IDENTIFIER : 86fbd746-cb12-41be-99c1-9d796bbb35ac
INODE : eeb8ca6b-982b-4c33-806e-d5023129faa5 IDENTIFIER : 9eeeee80-c42d-4915-aa22-02ae5080e266

Reindexing the structure didn't fix the issue.

Flush all caches didn't fix the issue either.

All

Comments

Work Log

Change History

Subversion Commits

FishEye

Sort Order:

Ascending order - Click to sort in descending order

[ Permlink | « Hide ]

Maria Bouza - July 29, 2010 1:09 PM

The only way to get this fixed was by manually removing the bad identifier using this script
http://web1.hca.dotcms.org/html/scripts/delete_from_index.jsp

[ Permlink | « Hide ]

Jason Tesser - July 29, 2010 1:20 PM

true or a full reindex.

[ Permlink | « Hide ]

Jason Tesser - July 29, 2010 1:21 PM

I think the destroy in the cms maintenance also would do this. It deletes from DB and deletes in index.

[ Permlink | « Hide ]

Maria Bouza - August 02, 2010 10:45 AM

JT,

This happened again to HCA this morning.

I'm going to manually fix it again, but we need to try to find out what's happening that adding bad stuff to the index.

Thanks
maru

[ Permlink | « Hide ]

Maria Bouza - August 04, 2010 12:05 PM

JT,

This happened again to HCA. :(

--Maru

[ Permlink | « Hide ]

Maria Bouza - August 04, 2010 12:12 PM

This time with this query:

+structureName:event +((event.startDate:[08/04/2010 00:00:00 TO 02/20/2011 00:00:00]) (+(event.startDate:[01/16/2010 00:00:00 TO 08/04/2010 00:00:00]) +(event.endDate:[08/04/2010 23:59:59 TO 02/20/2011 00:00:00]) )) +languageId:1 +deleted:false +live:true +conhost:32af50e2-1c69-45cb-b7da-a826e5dbf0c7*

I get 4 results in web1 and 3 results in web2.

Each one says they have 5 but they have bad identifiers.