Dropping sqlite column after data migration fails

[anonymous]

When following the tutorial at http://south.aeracode.org/docs/tutorial/part3.html the steps I took for replacing one field for another is:

• create the new field with a schema migration
• create a data migration to populate the new field
• drop the old field with a schema migration

When running the migration that drops the old field, south will crash with a traceback similar to the one below. This doesn't play nice with testing because running ./migrate test will fail every time.

I've attached a django app, boom, that reproduces the problem.

Thanks!

 > boom:0005_auto__del_field_boom_user
Traceback (most recent call last):
  File "./manage.py", line 11, in <module>
    execute_manager(settings)
  File "/Users/berto/local/opt/django/current/django/core/management/__init__.py", line 438, in execute_manager
    utility.execute()
  File "/Users/berto/local/opt/django/current/django/core/management/__init__.py", line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/berto/local/opt/django/current/django/core/management/base.py", line 191, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/Users/berto/local/opt/django/current/django/core/management/base.py", line 218, in execute
    output = self.handle(*args, **options)
  File "/Users/berto/Places/home/Projects/southboom/south/management/commands/migrate.py", line 105, in handle
    ignore_ghosts = ignore_ghosts,
  File "/Users/berto/Places/home/Projects/southboom/south/migration/__init__.py", line 191, in migrate_app
    success = migrator.migrate_many(target, workplan, database)
  File "/Users/berto/Places/home/Projects/southboom/south/migration/migrators.py", line 221, in migrate_many
    result = migrator.__class__.migrate_many(migrator, target, migrations, database)
  File "/Users/berto/Places/home/Projects/southboom/south/migration/migrators.py", line 292, in migrate_many
    result = self.migrate(migration, database)
  File "/Users/berto/Places/home/Projects/southboom/south/migration/migrators.py", line 125, in migrate
    result = self.run(migration)
  File "/Users/berto/Places/home/Projects/southboom/south/migration/migrators.py", line 99, in run
    return self.run_migration(migration)
  File "/Users/berto/Places/home/Projects/southboom/south/migration/migrators.py", line 81, in run_migration
    migration_function()
  File "/Users/berto/Places/home/Projects/southboom/south/migration/migrators.py", line 57, in <lambda>
    return (lambda: direction(orm))
  File "/Users/berto/Places/home/Projects/southboom/boom/migrations/0005_auto__del_field_boom_user.py", line 12, in forwards
    db.delete_column('boom_boom', 'user_id')
  File "/Users/berto/Places/home/Projects/southboom/south/db/sqlite3.py", line 135, in delete_column
    self._remake_table(table_name, deleted=[column_name])
  File "/Users/berto/Places/home/Projects/southboom/south/db/sqlite3.py", line 50, in _remake_table
    indexes = self._get_connection().introspection.get_indexes(cursor, table_name)
  File "/Users/berto/local/opt/django/current/django/db/backends/sqlite3/introspection.py", line 120, in get_indexes
    cursor.execute(query)
  File "/Users/berto/local/opt/django/current/django/db/backends/util.py", line 15, in execute
    return self.cursor.execute(sql, params)
  File "/Users/berto/local/opt/django/current/django/db/backends/sqlite3/base.py", line 200, in execute
    return Database.Cursor.execute(self, query, params)
IndexError: list index out of range

[anonymous]

django app to repro crash

[anonymous]

I've been trying to track down the source of this crash and the very same query that crashes, PRAGMA index_list('boom_boom'), executes just fine on the migration prior to the crashing one. Could the solution be reinitializing the database connection?

[andrew]

This works just great on my local install (using both 0.7.2 and the main development version). What OS, Python and SQLite version are you running?

[andrew]

Note: Related to bug #429 (possibly same issue)

[anonymous]

Hey Andrew,

I'm on Mac OS X Snow Leopard, python2.5.4 and its bundled version of sqlite3.

After you mentioned that it worked fine for you I decided to try with the OS X's python2.6 and the boom app does not go boom! I'm developing using python2.5 because that is what my production server is running. Ideally, this would work under python2.5, but having it work with python2.6 is making me consider upgrading to 2.6 now.

What versions of the software you asked about are you running? Is it feasible to see if this works with python2.5 on your end?

[anonymous]

Andrew,

It looks like the problem is with the sqlite module bundled with python 2.5. I downloaded and installed the latest sqlite version from the project page [1] and it is able to run the migrations without a problem.

Is this find worthy of a note on the main south page stating that anyone using south with sqlite on python2.5 should update their sqlite module?

Thanks for the help!

[1] ​http://code.google.com/p/pysqlite/downloads/detail?name=pysqlite-2.6.0.tar.gz&can=2&q=

[andrew]

Yeah, so this is a bug in that bundled version then. I've committed a docs update to the install guide in [ac5e8aa15052], and hopefully this ticket will tell other people that THEY NEED TO UPGRADE THEIR SQLITE MODULE.

Closing as invalid, since this is a bug in a third-party library.