Ticket #1222 (new defect)

Opened 22 months ago

Last modified 22 months ago

Index naming seems to have changed in Django 1.5, causes db.delete_index to fail

Reported by: strayer@… Owned by: andrew
Priority: major Milestone:
Component: migrations Version: 0.7.6
Keywords: Cc:

Description

I just upgraded to Django 1.5 and wanted to use the new index_together feature.

I deleted some db_index=True parameters and added the new Meta-option, then created the migration and (since it is not yet supported) added the index(_together) creation by hand. (I don't think the last part matters)

What happened now is that when migrating South tried to remove the old indexes (which is correct), but used a wrong name for them:

DatabaseError: (1091, "Can't DROP 'project_trade_81aefa79'; check that column/key exists")

When going back to Django 1.4.4 this same migration works like a charm. The suggested manual fixes of South clearly show the different names:

CREATE INDEX `project_tradedatadumps_3b1c9c31` ON `project_tradedatadumps` (`created_at`);
CREATE INDEX `project_trade_4d0541be` ON `project_trade` (`position`);
CREATE INDEX `project_trade_53af33f8` ON `project_trade` (`trade_state`);
CREATE INDEX `project_trade_55e41dff` ON `project_trade` (`formation`);
CREATE INDEX `project_trade_71e223e3` ON `project_trade` (`expires_at`);

CREATE INDEX `project_tradedatadumps_fde81f11` ON `project_tradedatadumps` (`created_at`);
CREATE INDEX `project_trade_4757fe07` ON `project_trade` (`position`);
CREATE INDEX `project_trade_8072a9de` ON `project_trade` (`trade_state`);
CREATE INDEX `project_trade_f1a41f23` ON `project_trade` (`formation`);
CREATE INDEX `project_trade_81aefa79` ON `project_trade` (`expires_at`);

After checking out how these names are generated (south/db/generic.py create_index_name) I noticed South is using the _digest function of BaseDatabaseCreation? - that one saw some changes for Python 3.3:
https://github.com/django/django/blame/1.5/django/db/backends/creation.py#L27
https://github.com/django/django/commit/e4ea53677449cfc56a0093bfbd92cb482020bb1e

"Ensued that SQL indexes are alwasy created in the same name.

Previous this used Python's builtin hash() function, which has never been guarnteed to be stable across implementations (CPython/Jython/etc.) or 32/64 bitness. However, this in practice it was stable. However, with the impending release of Python 3.3 hash randomizations is enabled by default, which would mean the index name changed between program invocations."

It looks like this is completely breaking all django-south applications that created indexes before Django 1.5 and try do remove them after upgrading to 1.5

I'll try to create a testcase project on GitHub? tomorrow, but I can't guarantee I'll find the time to do it.

Change History

comment:1 Changed 22 months ago by andrew

This is similar to #776. The fix is to have delete_index just scan the database for index names; everything else that deletes named items (e.g. deleting uniques) doesn't try to predict names.

comment:2 Changed 22 months ago by strayer@…

I tried to take a look at this today hoping it would be quite an easy fix, but it turns out this seems to be very specific to all the popular SQL servers and I can't just look at the INFORMATION_SCHEMA like in the delete_unique() function.

SQLite seems to be able to list index names, but not their fields.

MySQL and PostgreSQL should be able to do this, but they have their own commands to get an INDEX list.

Further reading:
PostgreSQL: http://www.postgresql.org/message-id/D960CB61B694CF459DCFB4B0128514C201E03810@exadv11.host.magwien.gv.at
MySQL: http://blog.9minutesnooze.com/mysql-information-schema-indexes/
SQLite: http://www.sqlite.org/faq.html#q7

Before spending more time on this I wanted to get back to you to ask if I'm on the wrong track or you have a better idea - I guess you have much more experience on this stuff than I do.

comment:3 Changed 22 months ago by andrew

No, that sounds about right (I've already written some of this code as part of my new Django migrations stuff - see https://github.com/django/django/pull/376)

SQLite probably doesn't support dropping indexes on tables anyway, it'd need the table to be re-created (like almost anything on that engine)

Note: See TracTickets for help on using tickets.