Ticket #604 (closed defect: invalid)

Opened 4 years ago

Last modified 4 years ago

Dropping sqlite column after data migration fails

Reported by: anonymous Owned by: andrew
Priority: major Milestone: 0.7.3
Component: commands Version: 0.7.2
Keywords: sqlite Cc: roberto.c.aguilar@…


When following the tutorial at http://south.aeracode.org/docs/tutorial/part3.html the steps I took for replacing one field for another is:

  • create the new field with a schema migration
  • create a data migration to populate the new field
  • drop the old field with a schema migration

When running the migration that drops the old field, south will crash with a traceback similar to the one below. This doesn't play nice with testing because running ./migrate test will fail every time.

I've attached a django app, boom, that reproduces the problem.


 > boom:0005_auto__del_field_boom_user
Traceback (most recent call last):
  File "./manage.py", line 11, in <module>
  File "/Users/berto/local/opt/django/current/django/core/management/__init__.py", line 438, in execute_manager
  File "/Users/berto/local/opt/django/current/django/core/management/__init__.py", line 379, in execute
  File "/Users/berto/local/opt/django/current/django/core/management/base.py", line 191, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/Users/berto/local/opt/django/current/django/core/management/base.py", line 218, in execute
    output = self.handle(*args, **options)
  File "/Users/berto/Places/home/Projects/southboom/south/management/commands/migrate.py", line 105, in handle
    ignore_ghosts = ignore_ghosts,
  File "/Users/berto/Places/home/Projects/southboom/south/migration/__init__.py", line 191, in migrate_app
    success = migrator.migrate_many(target, workplan, database)
  File "/Users/berto/Places/home/Projects/southboom/south/migration/migrators.py", line 221, in migrate_many
    result = migrator.__class__.migrate_many(migrator, target, migrations, database)
  File "/Users/berto/Places/home/Projects/southboom/south/migration/migrators.py", line 292, in migrate_many
    result = self.migrate(migration, database)
  File "/Users/berto/Places/home/Projects/southboom/south/migration/migrators.py", line 125, in migrate
    result = self.run(migration)
  File "/Users/berto/Places/home/Projects/southboom/south/migration/migrators.py", line 99, in run
    return self.run_migration(migration)
  File "/Users/berto/Places/home/Projects/southboom/south/migration/migrators.py", line 81, in run_migration
  File "/Users/berto/Places/home/Projects/southboom/south/migration/migrators.py", line 57, in <lambda>
    return (lambda: direction(orm))
  File "/Users/berto/Places/home/Projects/southboom/boom/migrations/0005_auto__del_field_boom_user.py", line 12, in forwards
    db.delete_column('boom_boom', 'user_id')
  File "/Users/berto/Places/home/Projects/southboom/south/db/sqlite3.py", line 135, in delete_column
    self._remake_table(table_name, deleted=[column_name])
  File "/Users/berto/Places/home/Projects/southboom/south/db/sqlite3.py", line 50, in _remake_table
    indexes = self._get_connection().introspection.get_indexes(cursor, table_name)
  File "/Users/berto/local/opt/django/current/django/db/backends/sqlite3/introspection.py", line 120, in get_indexes
  File "/Users/berto/local/opt/django/current/django/db/backends/util.py", line 15, in execute
    return self.cursor.execute(sql, params)
  File "/Users/berto/local/opt/django/current/django/db/backends/sqlite3/base.py", line 200, in execute
    return Database.Cursor.execute(self, query, params)
IndexError: list index out of range


boom.tar.gz (2.0 KB) - added by anonymous 4 years ago.
django app to repro crash

Change History

Changed 4 years ago by anonymous

django app to repro crash

comment:1 Changed 4 years ago by anonymous

I've been trying to track down the source of this crash and the very same query that crashes, PRAGMA index_list('boom_boom'), executes just fine on the migration prior to the crashing one. Could the solution be reinitializing the database connection?

comment:2 Changed 4 years ago by andrew

  • Status changed from new to infoneeded
  • Milestone set to 0.7.3

This works just great on my local install (using both 0.7.2 and the main development version). What OS, Python and SQLite version are you running?

comment:3 Changed 4 years ago by andrew

Note: Related to bug #429 (possibly same issue)

comment:4 Changed 4 years ago by anonymous

  • Status changed from infoneeded to assigned

Hey Andrew,

I'm on Mac OS X Snow Leopard, python2.5.4 and its bundled version of sqlite3.

After you mentioned that it worked fine for you I decided to try with the OS X's python2.6 and the boom app does not go boom! I'm developing using python2.5 because that is what my production server is running. Ideally, this would work under python2.5, but having it work with python2.6 is making me consider upgrading to 2.6 now.

What versions of the software you asked about are you running? Is it feasible to see if this works with python2.5 on your end?

comment:5 Changed 4 years ago by anonymous


It looks like the problem is with the sqlite module bundled with python 2.5. I downloaded and installed the latest sqlite version from the project page [1] and it is able to run the migrations without a problem.

Is this find worthy of a note on the main south page stating that anyone using south with sqlite on python2.5 should update their sqlite module?

Thanks for the help!

[1] http://code.google.com/p/pysqlite/downloads/detail?name=pysqlite-2.6.0.tar.gz&can=2&q=

comment:6 Changed 4 years ago by andrew

  • Status changed from assigned to closed
  • Resolution set to invalid

Yeah, so this is a bug in that bundled version then. I've committed a docs update to the install guide in [ac5e8aa15052], and hopefully this ticket will tell other people that THEY NEED TO UPGRADE THEIR SQLITE MODULE.

Closing as invalid, since this is a bug in a third-party library.

Note: See TracTickets for help on using tickets.