Ticket #1012 (closed defect: fixed)

Opened 3 years ago

Last modified 9 months ago

South Migrations fail with Django 1.4's Timezone Aware Feature

Reported by: cmsimike Owned by: andrew
Priority: major Milestone: 1.0
Component: migrations Version: mercurial
Keywords: Cc:

Description

Using Django 1.4 Alpha 1 with timezone awareness enabled causes migrations to fail.

The output from the migration attempt:
http://pastebin.com/8SRvq4uP

Attached is the migration file.

Attachments

0001_initial.py (13.1 KB) - added by anonymous 3 years ago.

Change History

Changed 3 years ago by anonymous

comment:1 Changed 3 years ago by andrew

  • Status changed from new to infoneeded
  • Milestone set to 1.0

Which version of South are you using?

comment:2 Changed 3 years ago by cmsimike

  • Status changed from infoneeded to assigned

0.7.3

comment:3 Changed 3 years ago by andrew

  • Status changed from assigned to infoneeded

Please retry with the latest development version, a fix was committed for this today.

comment:4 Changed 3 years ago by ryan.braley@…

  • Status changed from infoneeded to assigned

had this same issue and tried with the latest version of south https://bitbucket.org/andrewgodwin/south/ and I too have fields that look like
('django.db.models.fields.DateTimeField?', [], {'default': 'datetime.datetime(2012, 2, 20, 12, 54, 50, 95610, tzinfo=<django.utils.timezone.UTC object at 0x13fd050>)'})

The issue persists after that change was made

comment:5 Changed 3 years ago by indexofire@…

same issue in django1.4b1 + south7.3 and south dev version.

comment:6 Changed 3 years ago by indexofire@…

well after check the docs:
https://docs.djangoproject.com/en/dev/releases/1.4-beta-1/#serialization-of-datetime-and-time

just make USE_TZ = False if using SQLite as development database. then the DateTimeField? will restore to datetime.datetime.now

comment:7 Changed 3 years ago by jonathan.barratt@…

indexoffire's suggested fix of USE_TZ = False also worked for my postgres-backed Django 1.4-beta-1 installation. I had to rollback the migration that had been created before making that change to the project's settings (manually dropping tables), re-do schemamigration --initial and then I was good to go. In case it helps anyone else wondering if this fix is only good for SQLite...

comment:8 Changed 3 years ago by anonymous

This appears to be working with the latest tip (1022:fbc8b18d76e7, I think).

comment:9 Changed 3 years ago by andrew

Received by email reply from Jonathan:

That is the revision I was using when I encountered the TZ problem with Postgres:

jonathan@m2m-dev:/usr/local/lib/python2.7/dist-packages/south$ sudo hg summary
parent: 1022:fbc8b18d76e7 tip

If you're using postgres and aren't running into this problem, and haven't disabled Django's timezone awareness, all I can think of is that perhaps you're not declaring your datetime fields using "with time zone", e.g.: date_created timestamp(6) with time zone NOT NULL

I encountered the problem in both dev and staging environments, so it's definitely reproducible from my end...


I'll have to look into this myself, I think, as it's the only thing blocking a new release to coincide with Django 1.4.

comment:10 Changed 3 years ago by andrew

  • Status changed from assigned to closed
  • Resolution set to worksforme

I've just tried and I cannot replicate this problem with the latest Django release candidate and South commit. Unless someone can get me a model file which makes a bad migration (remember, already-created migrations will be bad, you'll need to make them with a new version of South), I'm leaving this closed.

comment:11 Changed 3 years ago by atkinsonr@…

I can replicate the problem using south-tip and Django 1.4 rc2! (current head)

Frozen pip includes:

-e git+https://github.com/django/django.git@62e6c1471da85336a9615d16fbde7cd6e7656e88#egg=Django-1.4c2-py2.6-dev
-e hg+https://bitbucket.org/andrewgodwin/south@286a7b6fb3d76a956cb317f25a9858e02919fe56#egg=South-tip
psycopg2==2.4.4

The error message I get when running schemamigration --auto ends with:
...
File "/home/rich/.virtualenvs/sd/src/south/south/orm.py", line 235, in eval_in_context

return eval(code, globals(), fake_locals)

File "<string>", line 1

SouthFieldClass?(default=datetime.datetime(2012, 3, 17, 2, 45, 9, 717141, tzinfo=<UTC>))

In settings I have USE_TZ = True

Note that I can do a schemamigration --initial, that works OK. It's only schemamigration --auto that produces the error.

Please let me know if I can help at all.

Rich

comment:12 Changed 3 years ago by atkinsonr@…

PS: Here is the whole error trace as it's more readable: https://gist.github.com/2054599

comment:13 Changed 3 years ago by atkinsonr@…

  • Status changed from closed to reopened
  • Resolution worksforme deleted

comment:14 Changed 3 years ago by atkinsonr@…

Oh, and nothing unusual in my model file, only date fields are:

created = models.DateTimeField?(auto_now_add=True)
updated = models.DateTimeField?(auto_now=True)

I'm pretty sure the steps to recreate are:

./manage.py schemamigration appname --initial
./manage.py migrate appname
.. make some change to your model
../manage.py schemamigration appname --auto

comment:15 Changed 3 years ago by andrew

  • Version changed from unknown to tip

Alright, I finally managed to replicate this, by inputting a datetime into the "you haven't specified a default" question that pops up - was this what you were doing? (as that only happens on --auto calls). If so, it's been fixed in [ab21b72cdf8a].

If not, I still can't replicate this.

comment:16 Changed 3 years ago by atkinsonr@…

Hello,
I'm pleased that you are able to recreate the issue, even if under slightly different circumstances to the way I am.

I'm not getting asked for a default as the only date fields I'm using include either auto_now=True or auto_now_add=True.

I wonder what other difference there could be?

My Python is 2.6.6 (Debian)

comment:17 Changed 3 years ago by andrew

I got prompted for a default even with auto_now enabled on a field (as it doesn't actually imply a database default). Have you tried with the latest commit I did anyway to see if it fixes it?

comment:18 Changed 3 years ago by atkinsonr@…

I didn't, but I should have because it appears to be fixed my condition also.

Thank you!

comment:19 Changed 3 years ago by andrew

  • Status changed from reopened to closed
  • Resolution set to fixed

comment:20 Changed 2 years ago by anonymous

  • Status changed from closed to reopened
  • Resolution fixed deleted

I have this problem too, see here:

http://pastebin.com/YGkhxbYV

Strangely, even when I remove or set USE_TZ=False in settings.py, retrying the migrations, it fails. I using south-tip installed from the mercurial repository. Please help!

comment:21 Changed 2 years ago by swihart.andrew@…

Forgot to fill in a name. Let me know if you need any other info. Also here'a migration that failed:

http://pastebin.com/zLwQxssb

comment:22 Changed 2 years ago by swihart.andrew@…

Another update -- I worked around the issue, realizing the previously failed migrations needed to be deleted. I though South would ignore them when I passed --initial, but I guess not, still need to learn how to use South correctly! Anyway, after deleting those schemamigration attempts, keeping USE_TZ=False, I was able to to an --auto schemamigration and migrate from my most recent successful migration.

I'm a little worn out right now to test this out again, but I'm pretty sure the USE_TZ=True gave South problems. I did add a couple DateTimeFields? in the interval that South had to deal with.

comment:23 Changed 2 years ago by andrew

  • Status changed from reopened to closed
  • Resolution set to fixed

The fix for this is in the migration generator, so you need to delete any and all migrations that have the error in them (an incorrectly coded datetime, so usually only the first one or two after you moved to 1.4/started a project) and then re-make them, as you've discovered. --initial doesn't reset everything, it just makes a new migration that will attempt to make everything again and fail.

Closing again until there's good proof this bug is still around (i.e. a brand new app that, when made and then changed, causes South to fail).

comment:24 Changed 2 years ago by justin@…

Ran into this one with a new app, with Django 1.4 and South 0.7.5. Here's the migration. Doing the sequence clean (generating and migrating) with USE_TZ=True gives the error:

<snipped>
File "<string>", line 1
    SouthFieldClass(default=datetime.datetime(2012, 7, 29, 15, 43, 3, 957057, tzinfo=<UTC>))
                                                                                     ^

Doing the sequence clean with USE_TZ=False and everything runs fine.

comment:25 Changed 2 years ago by andrew

Any chance of having the original model for that so I can try to reproduce it?

comment:26 Changed 2 years ago by justin@…

Sure:

class Item(models.Model):

    content = models.TextField(blank=True)
    creator = models.ForeignKey(User)
    created = models.DateTimeField(auto_now_add=True)
    last_updated = models.DateTimeField(auto_now=True)

That's the only model I've defined so far. Everything else is contrib or south.

comment:27 Changed 2 years ago by andrew

I've just made a migration for that same model and it works perfectly - in fact, there's no date at all in the field output, since auto_now_add is just kept as-is. Did you do anything weird to the core fields?

comment:28 Changed 2 years ago by justin@…

Nope, nothing weird. I'm just barely past setting up the whole project--adding the core app and Item model were my first actions. Everything besides my model is stock, off-the-shelf.

When I have USE_TZ = False in settings.py, it works perfectly; when USE_TZ = True, it fails with the error message I copied above (that initially led me to this bug).

A bit more data: It's not the date fields on my model, it's the default values for the date fields in the contrib User model listed in the models section. Here's the diff between the working and the non-working migration for my app:

[master] » diff f.py t.py 
44c44
<             'date_joined': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime(2012, 8, 2, 8, 33, 2, 254502)'}),
---
>             'date_joined': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime(2012, 8, 2, 15, 35, 57, 15428, tzinfo=<UTC>)'}),
52c52
<             'last_login': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime(2012, 8, 2, 8, 33, 2, 254432)'}),
---
>             'last_login': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime(2012, 8, 2, 15, 35, 57, 15353, tzinfo=<UTC>)'}),

It's the presence of tzinfo=<UTC> for the date_joined and last_login field defaults that's causing the migration to fail.

comment:29 Changed 2 years ago by justin@…

Here's my INSTALLED_APPS setting:

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'lists.core',
    'south',
)

comment:30 in reply to: ↑ description Changed 2 years ago by dp_wiz

quick'n'dirty fix without having to switch USE_TZ back and forth:

south/orm.py:
return eval(code.replace(', tzinfo=<UTC>))', '))'), globals(), fake_locals)

comment:31 Changed 2 years ago by andrew

I've just tried to reproduce this again and still can't make it fail. If someone with this issue can find me at DjangoCon? next week and demo it in front of me it'll make it a lot easier to debug - otherwise, something like an entire project zip would help.

comment:32 Changed 2 years ago by julianb

I had the same problem, but managed to fix it. I had an older version of South installed (presumably without timezone support) and then let it create a migration. This put the tzinfo=<UTC> stuff in my migration files. I then had the problem as described above, updated South and wondered why it still doesn't work. In my case I could delete the initial migration and redo it with the latest version and now everything works. So to sum it up:

  • If you only ever used the latest timezone compatible version of South you should not have this problem.
  • If you used an older version of South with Django 1.4 and timezones enabled, you probably have to fix your migrations.

comment:33 Changed 2 years ago by andrew

That sounds about right, Julian - South can't modify existing migrations. Unless any further evidence comes to light, I think this ticket should remain closed.

comment:34 Changed 2 years ago by peterkmurphy@…

Had the problem occur to me. TZ=True.

comment:35 Changed 2 years ago by pratikmandrekar@…

Had the problem with python 2.6.6, django 1.3 to django 1.4 migration with south 0.7.3 with USE_TZ=True.

Upgraded to South 0.7.6 and manually deleted the migration created post django migration from 1.3 to 1.4 that involved datetimefields. Worked great after that!

comment:36 Changed 9 months ago by anonymous

models = {

u'auth.group': {

'Meta': {'object_name': 'Group'},
u'id': ('django.db.models.fields.AutoField?', [], {'primary_key': 'True'}),
'name': ('django.db.models.fields.CharField?', [], {'unique': 'True', 'max_length': '80'}),
'permissions': ('django.db.models.fields.related.ManyToManyField?', [], {'to': u"ormauth.Permission?", 'symmetrical': 'False', 'blank': 'True'})

},
u'auth.permission': {

'Meta': {'ordering': "(u'content_typeapp_label', u'content_typemodel', u'codename')", 'unique_together': "((u'content_type', u'codename'),)", 'object_name': 'Permission'},
'codename': ('django.db.models.fields.CharField?', [], {'max_length': '100'}),
'content_type': ('django.db.models.fields.related.ForeignKey?', [], {'to': u"ormcontenttypes.ContentType?"}),
u'id': ('django.db.models.fields.AutoField?', [], {'primary_key': 'True'}),
'name': ('django.db.models.fields.CharField?', [], {'max_length': '50'})

},
u'auth.user': {

'Meta': {'object_name': 'User'},
'date_joined': ('django.db.models.fields.DateTimeField?', [], {'default': 'datetime.datetime(2014, 1, 15, 6, 38, 10, 599336, tzinfo=<UTC>)'}),
'email': ('django.db.models.fields.EmailField?', [], {'max_length': '75', 'blank': 'True'}),
'first_name': ('django.db.models.fields.CharField?', [], {'max_length': '30', 'blank': 'True'}),
'groups': ('django.db.models.fields.related.ManyToManyField?', [], {'symmetrical': 'False', 'related_name': "u'user_set'", 'blank': 'True', 'to': u"ormauth.Group?"}),
u'id': ('django.db.models.fields.AutoField?', [], {'primary_key': 'True'}),
'is_active': ('django.db.models.fields.BooleanField?', [], {'default': 'True'}),
'is_staff': ('django.db.models.fields.BooleanField?', [], {'default': 'False'}),
'is_superuser': ('django.db.models.fields.BooleanField?', [], {'default': 'False'}),
'last_login': ('django.db.models.fields.DateTimeField?', [], {'default': 'datetime.datetime(2014, 1, 15, 6, 38, 10, 598871, tzinfo=<UTC>)'}),
'last_name': ('django.db.models.fields.CharField?', [], {'max_length': '30', 'blank': 'True'}),
'password': ('django.db.models.fields.CharField?', [], {'max_length': '128'}),
'user_permissions': ('django.db.models.fields.related.ManyToManyField?', [], {'symmetrical': 'False', 'related_name': "u'user_set'", 'blank': 'True', 'to': u"ormauth.Permission?"}),
'username': ('django.db.models.fields.CharField?', [], {'unique': 'True', 'max_length': '30'})

},
u'contenttypes.contenttype': {

'Meta': {'ordering': "('name',)", 'unique_together': "(('app_label', 'model'),)", 'object_name': 'ContentType?', 'db_table': "'django_content_type'"},
'app_label': ('django.db.models.fields.CharField?', [], {'max_length': '100'}),
u'id': ('django.db.models.fields.AutoField?', [], {'primary_key': 'True'}),
'model': ('django.db.models.fields.CharField?', [], {'max_length': '100'}),
'name': ('django.db.models.fields.CharField?', [], {'max_length': '100'})

},
'profile.emailconfirmation': {

'Meta': {'object_name': 'EmailConfirmation?'},
u'id': ('django.db.models.fields.AutoField?', [], {'primary_key': 'True'}),
'key': ('django.db.models.fields.URLField', [], {'unique': 'True', 'max_length': '100'}),
'last_updated': ('django.db.models.fields.DateTimeField?', [], {'auto_now': 'True', 'blank': 'True'}),
'user': ('django.db.models.fields.related.OneToOneField?', [], {'to': u"ormauth.User?", 'unique': 'True'})

},
'profile.oauth2connection': {

'Meta': {'unique_together': "(('user', 'platform', 'profile_id'),)", 'object_name': 'OAuth2Connection'},
'access_expiration': ('django.db.models.fields.DateTimeField?', [], {'null': 'True', 'blank': 'True'}),
'access_token': ('django.db.models.fields.CharField?', [], {'unique': 'True', 'max_length': '200'}),
u'id': ('django.db.models.fields.AutoField?', [], {'primary_key': 'True'}),
'platform': ('django.db.models.fields.CharField?', [], {'max_length': '50'}),
'profile_id': ('django.db.models.fields.CharField?', [], {'max_length': '50'}),
'user': ('django.db.models.fields.related.ForeignKey?', [], {'blank': 'True', 'related_name': "'connections'", 'null': 'True', 'to': u"ormauth.User?"})

},
u'profile.userprofile': {

'Meta': {'object_name': 'UserProfile?'},
'age': ('django.db.models.fields.IntegerField?', [], {'max_length': '255', 'blank': 'True'}),
'body': ('django.db.models.fields.CharField?', [], {'max_length': '255', 'blank': 'True'}),
'body_structure': ('django.db.models.fields.CharField?', [], {'max_length': '255', 'blank': 'True'}),
'country': ('django.db.models.fields.CharField?', [], {'max_length': '255', 'blank': 'True'}),
'gender': ('django.db.models.fields.CharField?', [], {'max_length': '255', 'null': 'True', 'blank': 'True'}),
u'id': ('django.db.models.fields.AutoField?', [], {'primary_key': 'True'}),
'occupation': ('django.db.models.fields.CharField?', [], {'max_length': '255', 'blank': 'True'}),
'size': ('django.db.models.fields.CharField?', [], {'max_length': '255', 'blank': 'True'}),
'user': ('django.db.models.fields.related.OneToOneField?', [], {'to': u"ormauth.User?", 'unique': 'True'})

}

}

by commenting
datetime line where the

'last_login': ('django.db.models.fields.DateTimeField?', [], {'default': 'datetime.datetime(2014, 1, 15, 6, 38, 10, 598871, tzinfo=<UTC>)'}),

These lines
problem will be solved

Note: See TracTickets for help on using tickets.