Discussion:
[Community] Subject: Trying to add fields to 'properties' - invalid driver error
Ari Simmons
2014-05-29 23:37:50 UTC
Permalink
Okay...so, based on Sean's advice I circled back to the function *(thanks).
I am basically trying to COPY everything about one shapefile (data,
attributes, etc) into another shapefile, with one addition - a new
attribute field (i.e. 'shield_type').

However all attempts to do this result in a 'driver error'....(?)

Here is what I got:

import fiona
from shapely.geometry import shape, mapping

def process_file(self, inFile, outFile):
with fiona.collection(inFile, 'r') as input:
input_schema = input.schema
input_schema['geometry'] = 'LineString'
attributes = dict(input_schema['properties'])
attributes.update({'shield_type':'str:1'})
input_schema['properties'] = attributes
with fiona.collection(outFile, 'w', input_schema) as output:
for elem in input:
output.write({'properties':
elem['properties'],'geometry': mapping(shape(elem['geometry']))})
g = ShieldLabels()
inFile =
r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data.shp'
outFile = r'I:\It_24\115507_Road_Shields_Label_Processing\data\ari.shp'
g.process_file(inFile, outFile)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File
"I:/It_24/115507_Road_Shields_Label_Processing/code/RoadShieldLabels/ShieldLabels.py",
line 23, in process_file
with fiona.collection(outFile, 'w', input_schema) as output:
File "C:\Python27\lib\site-packages\fiona\__init__.py", line 147, in open
encoding=encoding, layer=layer, vsi=vsi, archive=archive)
File "C:\Python27\lib\site-packages\fiona\collection.py", line 42, in
__init__
raise TypeError("invalid driver: %r" % driver)
TypeError: invalid driver: {'geometry': 'LineString', 'properties':
{u'Z_ORDER': 'int:6', u'BRIDGE': 'int:6', u'NAME': 'str:254', u'TUNNEL':
'int:6', u'TYPE': 'str:254', u'OSM_ID': 'float:19', 'shield_type': 'str:1',
u'ONEWAY': 'int:6', u'REF': 'str:254', u'ID': 'float:11'}}
Message: 1
Date: Wed, 28 May 2014 15:04:06 -0700
From: Ari Simmons <ari.ucb.fire at gmail.com>
Subject: [Community] Trying to add fields using meta
To: "community at lists.gispython.org" <community at lists.gispython.org>
<
CAJxqeBsRh1GgpD9OBx01_i_UtY0cWKGwqfnOUyo4RxEh1RYUbw at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
I am trying to copy the schema of an existing shapefile and add to it in an
output shapefile. Somehow I am not getting anywhere with this...though the
https://pypi.python.org/pypi/Fiona) and it is just adding to a dictionary.
For this object, I have this schema
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data.shp')
c.schema
{'geometry': 'LineString', 'properties': OrderedDict([(u'ID', 'float:11'),
(u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE', 'str:254'),
(u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY', 'int:6'), (u'REF',
'str:254'), (u'Z_ORDER', 'int:6')])}
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp')
c.schema
{'geometry': 'LineString', 'properties': OrderedDict([(u'ID', 'float:11'),
(u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE', 'str:254'),
(u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY', 'int:6'), (u'REF',
'str:254'), (u'Z_ORDER', 'int:6'), (u'shield_type', 'str:254'), (u'label',
'str:254'), (u'label_len', 'int:10'), (u'zoom', 'int:10')])}
meta = input.meta
# create new fields for the new schema
meta['schema']['properties']['shield_type'.encode("utf-8")] =
'str:254'
meta['schema']['properties']['label'.encode("utf-8")] =
'str:254'
meta['schema']['properties']['label_len'] = 'int:10'
meta['schema']['properties']['zoom'] = 'int:10'
n = item.copy()
new_data_attributes =
function_using_some_regex_parsing(item['properties']['REF']
...
Right now running this I just get an empty dictionary...I'm not sure what
I'm missing...
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp')
c.schema
{'geometry': 'LineString', 'properties': OrderedDict()}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://lists.gispython.org/pipermail/community/attachments/20140528/2788d117/attachment.html
------------------------------
Message: 2
Date: Wed, 28 May 2014 22:29:39 -0600
From: Sean Gillies <sean.gillies at gmail.com>
Subject: Re: [Community] Trying to add fields using meta
To: "gispython.org community projects" <community at lists.gispython.org>
<
CAOodmJpBb34SBNkjkXT-CXEtNgNiH1zPb7jZeTcC1KWJ00hMCg at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi Ari,
Is there any chance that the
I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp isn't
being written properly due to some logic in your function? My code below
$ python
Python 2.7.6 (default, May 8 2014, 07:38:16)
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
import fiona
from collections import OrderedDict
schema = {'geometry': 'LineString', 'properties': OrderedDict([(u'ID',
'float:11'), (u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE',
'str:254'), (u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY',
'int:6'), (u'REF', 'str:254'), (u'Z_ORDER', 'int:6'), (u'shield_type',
'str:254'), (u'label', 'str:254'), (u'label_len', 'int:10'), (u'zoom',
'int:10')])}
with fiona.open('/tmp/foo.shp', 'w', crs={'init': 'epsg:4326'},
... pass
...
import subprocess
subprocess.check_output(['ogrinfo', '/tmp/foo.shp', 'foo'])
'INFO: Open of `/tmp/foo.shp\'\n using driver `ESRI Shapefile\'
0\nExtent: (0.000000, 0.000000) - (0.000000, 0.000000)\nLayer SRS
WKT:\nGEOGCS["GCS_WGS_1984",\n DATUM["WGS_1984",\n
SPHEROID["WGS_84",6378137,298.257223563]],\n PRIMEM["Greenwich",0],\n
UNIT["Degree",0.017453292519943295]]\nID: Real (11.0)\nOSM_ID: Real
(19.0)\nNAME: String (254.0)\nTYPE: String (254.0)\nTUNNEL: Integer
(6.0)\nBRIDGE: Integer (6.0)\nONEWAY: Integer (6.0)\nREF: String
(254.0)\nZ_ORDER: Integer (6.0)\nshield_typ: String (254.0)\nlabel: String
(254.0)\nlabel_len: Integer (10.0)\nzoom: Integer (10.0)\n'
On Wed, May 28, 2014 at 4:04 PM, Ari Simmons <ari.ucb.fire at gmail.com>
I am trying to copy the schema of an existing shapefile and add to it in
an output shapefile. Somehow I am not getting anywhere with this...though
https://pypi.python.org/pypi/Fiona) and it is just adding to a
dictionary.
For this object, I have this schema
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data.shp')
c.schema
{'geometry': 'LineString', 'properties': OrderedDict([(u'ID',
'float:11'),
(u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE', 'str:254'),
(u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY', 'int:6'),
(u'REF',
'str:254'), (u'Z_ORDER', 'int:6')])}
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp')
c.schema
{'geometry': 'LineString', 'properties': OrderedDict([(u'ID',
'float:11'),
(u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE', 'str:254'),
(u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY', 'int:6'),
(u'REF',
'str:254'), (u'Z_ORDER', 'int:6'), (u'shield_type', 'str:254'),
(u'label',
'str:254'), (u'label_len', 'int:10'), (u'zoom', 'int:10')])}
meta = input.meta
# create new fields for the new schema
meta['schema']['properties']['shield_type'.encode("utf-8")] =
'str:254'
meta['schema']['properties']['label'.encode("utf-8")] =
'str:254'
meta['schema']['properties']['label_len'] = 'int:10'
meta['schema']['properties']['zoom'] = 'int:10'
n = item.copy()
new_data_attributes =
function_using_some_regex_parsing(item['properties']['REF']
...
Right now running this I just get an empty dictionary...I'm not sure what
I'm missing...
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp')
c.schema
{'geometry': 'LineString', 'properties': OrderedDict()}
_______________________________________________
Community mailing list
Community at lists.gispython.org
http://lists.gispython.org/mailman/listinfo/community
--
Sean Gillies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://lists.gispython.org/pipermail/community/attachments/20140528/8908242a/attachment-0001.htm
------------------------------
_______________________________________________
Community mailing list
Community at lists.gispython.org
http://lists.gispython.org/mailman/listinfo/community
End of Community Digest, Vol 98, Issue 8
****************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gispython.org/pipermail/community/attachments/20140529/9420cd60/attachment.htm>
Ari Simmons
2014-05-29 23:54:05 UTC
Permalink
ok. I see the problem it is in write()...

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File
"I:/It_24/115507_Road_Shields_Label_Processing/code/RoadShieldLabels/ShieldLabels.py",
line 24, in process_file
output.write({'properties': elem['properties'],'geometry':
mapping(shape(elem['geometry']))})
File "C:\Python27\lib\site-packages\fiona\collection.py", line 211, in
write
self.writerecords([record])
File "C:\Python27\lib\site-packages\fiona\collection.py", line 205, in
writerecords
self.session.writerecs(records, self)
File "ogrext.pyx", line 987, in fiona.ogrext.WritingSession.writerecs
(src/fiona/ogrext.c:15807)
ValueError: Record does not match collection schema: [u'ID', u'OSM_ID',
u'NAME', u'TYPE', u'TUNNEL', u'BRIDGE', u'ONEWAY', u'REF', u'Z_ORDER'] !=
[u'Z_ORDER', u'BRIDGE', u'NAME', u'TUNNEL', u'REF', u'ONEWAY', 'shield_ty',
u'OSM_ID', u'TYPE', u'ID']
Post by Ari Simmons
Okay...so, based on Sean's advice I circled back to the function
*(thanks). I am basically trying to COPY everything about one shapefile
(data, attributes, etc) into another shapefile, with one addition - a new
attribute field (i.e. 'shield_type').
However all attempts to do this result in a 'driver error'....(?)
import fiona
from shapely.geometry import shape, mapping
input_schema = input.schema
input_schema['geometry'] = 'LineString'
attributes = dict(input_schema['properties'])
attributes.update({'shield_type':'str:1'})
input_schema['properties'] = attributes
elem['properties'],'geometry': mapping(shape(elem['geometry']))})
g = ShieldLabels()
inFile =
r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data.shp'
outFile = r'I:\It_24\115507_Road_Shields_Label_Processing\data\ari.shp'
g.process_file(inFile, outFile)
File "<stdin>", line 1, in <module>
File
"I:/It_24/115507_Road_Shields_Label_Processing/code/RoadShieldLabels/ShieldLabels.py",
line 23, in process_file
File "C:\Python27\lib\site-packages\fiona\__init__.py", line 147, in open
encoding=encoding, layer=layer, vsi=vsi, archive=archive)
File "C:\Python27\lib\site-packages\fiona\collection.py", line 42, in
__init__
raise TypeError("invalid driver: %r" % driver)
'int:6', u'TYPE': 'str:254', u'OSM_ID': 'float:19', 'shield_type': 'str:1',
u'ONEWAY': 'int:6', u'REF': 'str:254', u'ID': 'float:11'}}
Message: 1
Date: Wed, 28 May 2014 15:04:06 -0700
From: Ari Simmons <ari.ucb.fire at gmail.com>
Subject: [Community] Trying to add fields using meta
To: "community at lists.gispython.org" <community at lists.gispython.org>
<
CAJxqeBsRh1GgpD9OBx01_i_UtY0cWKGwqfnOUyo4RxEh1RYUbw at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
I am trying to copy the schema of an existing shapefile and add to it in an
output shapefile. Somehow I am not getting anywhere with this...though the
https://pypi.python.org/pypi/Fiona) and it is just adding to a dictionary.
For this object, I have this schema
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data.shp')
c.schema
{'geometry': 'LineString', 'properties': OrderedDict([(u'ID', 'float:11'),
(u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE', 'str:254'),
(u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY', 'int:6'), (u'REF',
'str:254'), (u'Z_ORDER', 'int:6')])}
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp')
c.schema
{'geometry': 'LineString', 'properties': OrderedDict([(u'ID', 'float:11'),
(u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE', 'str:254'),
(u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY', 'int:6'), (u'REF',
'str:254'), (u'Z_ORDER', 'int:6'), (u'shield_type', 'str:254'), (u'label',
'str:254'), (u'label_len', 'int:10'), (u'zoom', 'int:10')])}
meta = input.meta
# create new fields for the new schema
meta['schema']['properties']['shield_type'.encode("utf-8")] =
'str:254'
meta['schema']['properties']['label'.encode("utf-8")] =
'str:254'
meta['schema']['properties']['label_len'] = 'int:10'
meta['schema']['properties']['zoom'] = 'int:10'
n = item.copy()
new_data_attributes =
function_using_some_regex_parsing(item['properties']['REF']
...
Right now running this I just get an empty dictionary...I'm not sure what
I'm missing...
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp')
c.schema
{'geometry': 'LineString', 'properties': OrderedDict()}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://lists.gispython.org/pipermail/community/attachments/20140528/2788d117/attachment.html
------------------------------
Message: 2
Date: Wed, 28 May 2014 22:29:39 -0600
From: Sean Gillies <sean.gillies at gmail.com>
Subject: Re: [Community] Trying to add fields using meta
To: "gispython.org community projects" <community at lists.gispython.org>
<
CAOodmJpBb34SBNkjkXT-CXEtNgNiH1zPb7jZeTcC1KWJ00hMCg at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi Ari,
Is there any chance that the
I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp isn't
being written properly due to some logic in your function? My code below
$ python
Python 2.7.6 (default, May 8 2014, 07:38:16)
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
import fiona
from collections import OrderedDict
schema = {'geometry': 'LineString', 'properties': OrderedDict([(u'ID',
'float:11'), (u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE',
'str:254'), (u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY',
'int:6'), (u'REF', 'str:254'), (u'Z_ORDER', 'int:6'), (u'shield_type',
'str:254'), (u'label', 'str:254'), (u'label_len', 'int:10'), (u'zoom',
'int:10')])}
with fiona.open('/tmp/foo.shp', 'w', crs={'init': 'epsg:4326'},
... pass
...
import subprocess
subprocess.check_output(['ogrinfo', '/tmp/foo.shp', 'foo'])
'INFO: Open of `/tmp/foo.shp\'\n using driver `ESRI Shapefile\'
0\nExtent: (0.000000, 0.000000) - (0.000000, 0.000000)\nLayer SRS
WKT:\nGEOGCS["GCS_WGS_1984",\n DATUM["WGS_1984",\n
SPHEROID["WGS_84",6378137,298.257223563]],\n PRIMEM["Greenwich",0],\n
UNIT["Degree",0.017453292519943295]]\nID: Real (11.0)\nOSM_ID: Real
(19.0)\nNAME: String (254.0)\nTYPE: String (254.0)\nTUNNEL: Integer
(6.0)\nBRIDGE: Integer (6.0)\nONEWAY: Integer (6.0)\nREF: String
(254.0)\nZ_ORDER: Integer (6.0)\nshield_typ: String (254.0)\nlabel: String
(254.0)\nlabel_len: Integer (10.0)\nzoom: Integer (10.0)\n'
On Wed, May 28, 2014 at 4:04 PM, Ari Simmons <ari.ucb.fire at gmail.com>
I am trying to copy the schema of an existing shapefile and add to it in
an output shapefile. Somehow I am not getting anywhere with
this...though
https://pypi.python.org/pypi/Fiona) and it is just adding to a
dictionary.
For this object, I have this schema
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data.shp')
c.schema
{'geometry': 'LineString', 'properties': OrderedDict([(u'ID',
'float:11'),
(u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE', 'str:254'),
(u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY', 'int:6'),
(u'REF',
'str:254'), (u'Z_ORDER', 'int:6')])}
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp')
c.schema
{'geometry': 'LineString', 'properties': OrderedDict([(u'ID',
'float:11'),
(u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE', 'str:254'),
(u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY', 'int:6'),
(u'REF',
'str:254'), (u'Z_ORDER', 'int:6'), (u'shield_type', 'str:254'),
(u'label',
'str:254'), (u'label_len', 'int:10'), (u'zoom', 'int:10')])}
meta = input.meta
# create new fields for the new schema
meta['schema']['properties']['shield_type'.encode("utf-8")]
=
'str:254'
meta['schema']['properties']['label'.encode("utf-8")] =
'str:254'
meta['schema']['properties']['label_len'] = 'int:10'
meta['schema']['properties']['zoom'] = 'int:10'
n = item.copy()
new_data_attributes =
function_using_some_regex_parsing(item['properties']['REF']
...
Right now running this I just get an empty dictionary...I'm not sure
what
I'm missing...
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp')
c.schema
{'geometry': 'LineString', 'properties': OrderedDict()}
_______________________________________________
Community mailing list
Community at lists.gispython.org
http://lists.gispython.org/mailman/listinfo/community
--
Sean Gillies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://lists.gispython.org/pipermail/community/attachments/20140528/8908242a/attachment-0001.htm
------------------------------
_______________________________________________
Community mailing list
Community at lists.gispython.org
http://lists.gispython.org/mailman/listinfo/community
End of Community Digest, Vol 98, Issue 8
****************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gispython.org/pipermail/community/attachments/20140529/97aa145a/attachment-0001.htm>
Sean Gillies
2014-06-01 22:54:37 UTC
Permalink
Ari,

I think the problem is that "shield_type" is 11 characters ? too long for a
shapefile field, and so OGR (and FIona) tuncates it to "shield_typ". In
Fiona versions before 1.1.4, an attempt to write a feature with a
"shield_type" property would fail. In version 1.1.4, it should work thanks
to an internal mapping. Can you upgrade or confirm that you have version
1.1.4+ and that this is a bug?
Post by Ari Simmons
ok. I see the problem it is in write()...
File "<stdin>", line 1, in <module>
File
"I:/It_24/115507_Road_Shields_Label_Processing/code/RoadShieldLabels/ShieldLabels.py",
line 24, in process_file
mapping(shape(elem['geometry']))})
File "C:\Python27\lib\site-packages\fiona\collection.py", line 211, in
write
self.writerecords([record])
File "C:\Python27\lib\site-packages\fiona\collection.py", line 205, in
writerecords
self.session.writerecs(records, self)
File "ogrext.pyx", line 987, in fiona.ogrext.WritingSession.writerecs
(src/fiona/ogrext.c:15807)
ValueError: Record does not match collection schema: [u'ID', u'OSM_ID',
u'NAME', u'TYPE', u'TUNNEL', u'BRIDGE', u'ONEWAY', u'REF', u'Z_ORDER'] !=
[u'Z_ORDER', u'BRIDGE', u'NAME', u'TUNNEL', u'REF', u'ONEWAY', 'shield_ty',
u'OSM_ID', u'TYPE', u'ID']
Post by Ari Simmons
Okay...so, based on Sean's advice I circled back to the function
*(thanks). I am basically trying to COPY everything about one shapefile
(data, attributes, etc) into another shapefile, with one addition - a new
attribute field (i.e. 'shield_type').
However all attempts to do this result in a 'driver error'....(?)
import fiona
from shapely.geometry import shape, mapping
input_schema = input.schema
input_schema['geometry'] = 'LineString'
attributes = dict(input_schema['properties'])
attributes.update({'shield_type':'str:1'})
input_schema['properties'] = attributes
elem['properties'],'geometry': mapping(shape(elem['geometry']))})
g = ShieldLabels()
inFile =
r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data.shp'
outFile =
r'I:\It_24\115507_Road_Shields_Label_Processing\data\ari.shp'
g.process_file(inFile, outFile)
File "<stdin>", line 1, in <module>
File
"I:/It_24/115507_Road_Shields_Label_Processing/code/RoadShieldLabels/ShieldLabels.py",
line 23, in process_file
File "C:\Python27\lib\site-packages\fiona\__init__.py", line 147, in open
encoding=encoding, layer=layer, vsi=vsi, archive=archive)
File "C:\Python27\lib\site-packages\fiona\collection.py", line 42, in
__init__
raise TypeError("invalid driver: %r" % driver)
'int:6', u'TYPE': 'str:254', u'OSM_ID': 'float:19', 'shield_type': 'str:1',
u'ONEWAY': 'int:6', u'REF': 'str:254', u'ID': 'float:11'}}
Message: 1
Date: Wed, 28 May 2014 15:04:06 -0700
From: Ari Simmons <ari.ucb.fire at gmail.com>
Subject: [Community] Trying to add fields using meta
To: "community at lists.gispython.org" <community at lists.gispython.org>
<
CAJxqeBsRh1GgpD9OBx01_i_UtY0cWKGwqfnOUyo4RxEh1RYUbw at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
I am trying to copy the schema of an existing shapefile and add to it in an
output shapefile. Somehow I am not getting anywhere with this...though the
https://pypi.python.org/pypi/Fiona) and it is just adding to a dictionary.
For this object, I have this schema
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data.shp')
c.schema
{'geometry': 'LineString', 'properties': OrderedDict([(u'ID', 'float:11'),
(u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE', 'str:254'),
(u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY', 'int:6'), (u'REF',
'str:254'), (u'Z_ORDER', 'int:6')])}
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp')
c.schema
{'geometry': 'LineString', 'properties': OrderedDict([(u'ID', 'float:11'),
(u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE', 'str:254'),
(u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY', 'int:6'), (u'REF',
'str:254'), (u'Z_ORDER', 'int:6'), (u'shield_type', 'str:254'), (u'label',
'str:254'), (u'label_len', 'int:10'), (u'zoom', 'int:10')])}
meta = input.meta
# create new fields for the new schema
meta['schema']['properties']['shield_type'.encode("utf-8")] =
'str:254'
meta['schema']['properties']['label'.encode("utf-8")] =
'str:254'
meta['schema']['properties']['label_len'] = 'int:10'
meta['schema']['properties']['zoom'] = 'int:10'
n = item.copy()
new_data_attributes =
function_using_some_regex_parsing(item['properties']['REF']
...
Right now running this I just get an empty dictionary...I'm not sure what
I'm missing...
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp')
c.schema
{'geometry': 'LineString', 'properties': OrderedDict()}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://lists.gispython.org/pipermail/community/attachments/20140528/2788d117/attachment.html
------------------------------
Message: 2
Date: Wed, 28 May 2014 22:29:39 -0600
From: Sean Gillies <sean.gillies at gmail.com>
Subject: Re: [Community] Trying to add fields using meta
To: "gispython.org community projects" <community at lists.gispython.org>
<
CAOodmJpBb34SBNkjkXT-CXEtNgNiH1zPb7jZeTcC1KWJ00hMCg at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi Ari,
Is there any chance that the
I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp isn't
being written properly due to some logic in your function? My code below
$ python
Python 2.7.6 (default, May 8 2014, 07:38:16)
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
import fiona
from collections import OrderedDict
OrderedDict([(u'ID',
'float:11'), (u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE',
'str:254'), (u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY',
'int:6'), (u'REF', 'str:254'), (u'Z_ORDER', 'int:6'), (u'shield_type',
'str:254'), (u'label', 'str:254'), (u'label_len', 'int:10'), (u'zoom',
'int:10')])}
with fiona.open('/tmp/foo.shp', 'w', crs={'init': 'epsg:4326'},
... pass
...
import subprocess
subprocess.check_output(['ogrinfo', '/tmp/foo.shp', 'foo'])
'INFO: Open of `/tmp/foo.shp\'\n using driver `ESRI Shapefile\'
0\nExtent: (0.000000, 0.000000) - (0.000000, 0.000000)\nLayer SRS
WKT:\nGEOGCS["GCS_WGS_1984",\n DATUM["WGS_1984",\n
SPHEROID["WGS_84",6378137,298.257223563]],\n PRIMEM["Greenwich",0],\n
UNIT["Degree",0.017453292519943295]]\nID: Real (11.0)\nOSM_ID: Real
(19.0)\nNAME: String (254.0)\nTYPE: String (254.0)\nTUNNEL: Integer
(6.0)\nBRIDGE: Integer (6.0)\nONEWAY: Integer (6.0)\nREF: String
(254.0)\nZ_ORDER: Integer (6.0)\nshield_typ: String (254.0)\nlabel: String
(254.0)\nlabel_len: Integer (10.0)\nzoom: Integer (10.0)\n'
On Wed, May 28, 2014 at 4:04 PM, Ari Simmons <ari.ucb.fire at gmail.com>
I am trying to copy the schema of an existing shapefile and add to it
in
an output shapefile. Somehow I am not getting anywhere with
this...though
https://pypi.python.org/pypi/Fiona) and it is just adding to a
dictionary.
For this object, I have this schema
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data.shp')
c.schema
{'geometry': 'LineString', 'properties': OrderedDict([(u'ID',
'float:11'),
(u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE', 'str:254'),
(u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY', 'int:6'),
(u'REF',
'str:254'), (u'Z_ORDER', 'int:6')])}
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp')
c.schema
{'geometry': 'LineString', 'properties': OrderedDict([(u'ID',
'float:11'),
(u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE', 'str:254'),
(u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY', 'int:6'),
(u'REF',
'str:254'), (u'Z_ORDER', 'int:6'), (u'shield_type', 'str:254'),
(u'label',
'str:254'), (u'label_len', 'int:10'), (u'zoom', 'int:10')])}
meta = input.meta
# create new fields for the new schema
meta['schema']['properties']['shield_type'.encode("utf-8")] =
'str:254'
meta['schema']['properties']['label'.encode("utf-8")] =
'str:254'
meta['schema']['properties']['label_len'] = 'int:10'
meta['schema']['properties']['zoom'] = 'int:10'
n = item.copy()
new_data_attributes =
function_using_some_regex_parsing(item['properties']['REF']
...
Right now running this I just get an empty dictionary...I'm not sure
what
I'm missing...
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp')
c.schema
{'geometry': 'LineString', 'properties': OrderedDict()}
_______________________________________________
Community mailing list
Community at lists.gispython.org
http://lists.gispython.org/mailman/listinfo/community
--
Sean Gillies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://lists.gispython.org/pipermail/community/attachments/20140528/8908242a/attachment-0001.htm
------------------------------
_______________________________________________
Community mailing list
Community at lists.gispython.org
http://lists.gispython.org/mailman/listinfo/community
End of Community Digest, Vol 98, Issue 8
****************************************
_______________________________________________
Community mailing list
Community at lists.gispython.org
http://lists.gispython.org/mailman/listinfo/community
--
Sean Gillies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gispython.org/pipermail/community/attachments/20140601/2c0f2f9b/attachment-0001.htm>
Ari Simmons
2014-06-02 20:21:36 UTC
Permalink
Hi Sean,

I think you are right....went down to 'shield_ty' and it worked. I am
running fiona 1.0.2..I can ugrade to 1.1.5 and test, or do you want me to
try 1.1.4 specifically?

-Ari

On Sun, Jun 1, 2014 at 3:54 PM, <community-request at lists.gispython.org>
Send Community mailing list submissions to
community at lists.gispython.org
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.gispython.org/mailman/listinfo/community
or, via email, send a message with subject or body 'help' to
community-request at lists.gispython.org
You can reach the person managing the list at
community-owner at lists.gispython.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Community digest..."
1. Re: Subject: Trying to add fields to 'properties' - invalid
driver error (Sean Gillies)
----------------------------------------------------------------------
Message: 1
Date: Sun, 1 Jun 2014 16:54:37 -0600
From: Sean Gillies <sean.gillies at gmail.com>
Subject: Re: [Community] Subject: Trying to add fields to 'properties'
- invalid driver error
To: "gispython.org community projects" <community at lists.gispython.org>
<CAOodmJo=O=
m_ZXmcOT2DRSYxbUxyJyeSetZ9F0ETDOHj4a1FLA at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Ari,
I think the problem is that "shield_type" is 11 characters ? too long for a
shapefile field, and so OGR (and FIona) tuncates it to "shield_typ". In
Fiona versions before 1.1.4, an attempt to write a feature with a
"shield_type" property would fail. In version 1.1.4, it should work thanks
to an internal mapping. Can you upgrade or confirm that you have version
1.1.4+ and that this is a bug?
On Thu, May 29, 2014 at 5:54 PM, Ari Simmons <ari.ucb.fire at gmail.com>
Post by Ari Simmons
ok. I see the problem it is in write()...
File "<stdin>", line 1, in <module>
File
"I:/It_24/115507_Road_Shields_Label_Processing/code/RoadShieldLabels/ShieldLabels.py",
Post by Ari Simmons
line 24, in process_file
mapping(shape(elem['geometry']))})
File "C:\Python27\lib\site-packages\fiona\collection.py", line 211, in
write
self.writerecords([record])
File "C:\Python27\lib\site-packages\fiona\collection.py", line 205, in
writerecords
self.session.writerecs(records, self)
File "ogrext.pyx", line 987, in fiona.ogrext.WritingSession.writerecs
(src/fiona/ogrext.c:15807)
ValueError: Record does not match collection schema: [u'ID', u'OSM_ID',
u'NAME', u'TYPE', u'TUNNEL', u'BRIDGE', u'ONEWAY', u'REF', u'Z_ORDER'] !=
[u'Z_ORDER', u'BRIDGE', u'NAME', u'TUNNEL', u'REF', u'ONEWAY',
'shield_ty',
Post by Ari Simmons
u'OSM_ID', u'TYPE', u'ID']
On Thu, May 29, 2014 at 4:37 PM, Ari Simmons <ari.ucb.fire at gmail.com>
Post by Ari Simmons
Okay...so, based on Sean's advice I circled back to the function
*(thanks). I am basically trying to COPY everything about one shapefile
(data, attributes, etc) into another shapefile, with one addition - a
new
Post by Ari Simmons
Post by Ari Simmons
attribute field (i.e. 'shield_type').
However all attempts to do this result in a 'driver error'....(?)
import fiona
from shapely.geometry import shape, mapping
input_schema = input.schema
input_schema['geometry'] = 'LineString'
attributes = dict(input_schema['properties'])
attributes.update({'shield_type':'str:1'})
input_schema['properties'] = attributes
elem['properties'],'geometry': mapping(shape(elem['geometry']))})
g = ShieldLabels()
inFile =
r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data.shp'
outFile =
r'I:\It_24\115507_Road_Shields_Label_Processing\data\ari.shp'
g.process_file(inFile, outFile)
File "<stdin>", line 1, in <module>
File
"I:/It_24/115507_Road_Shields_Label_Processing/code/RoadShieldLabels/ShieldLabels.py",
Post by Ari Simmons
Post by Ari Simmons
line 23, in process_file
File "C:\Python27\lib\site-packages\fiona\__init__.py", line 147, in
open
encoding=encoding, layer=layer, vsi=vsi, archive=archive)
File "C:\Python27\lib\site-packages\fiona\collection.py", line 42, in
__init__
raise TypeError("invalid driver: %r" % driver)
'str:1',
Post by Ari Simmons
Post by Ari Simmons
u'ONEWAY': 'int:6', u'REF': 'str:254', u'ID': 'float:11'}}
Message: 1
Date: Wed, 28 May 2014 15:04:06 -0700
From: Ari Simmons <ari.ucb.fire at gmail.com>
Subject: [Community] Trying to add fields using meta
To: "community at lists.gispython.org" <community at lists.gispython.org>
<
CAJxqeBsRh1GgpD9OBx01_i_UtY0cWKGwqfnOUyo4RxEh1RYUbw at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
I am trying to copy the schema of an existing shapefile and add to it
in
Post by Ari Simmons
Post by Ari Simmons
an
output shapefile. Somehow I am not getting anywhere with this...though
the
https://pypi.python.org/pypi/Fiona) and it is just adding to a
dictionary.
For this object, I have this schema
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data.shp')
Post by Ari Simmons
Post by Ari Simmons
c.schema
{'geometry': 'LineString', 'properties': OrderedDict([(u'ID',
'float:11'),
(u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE', 'str:254'),
(u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY', 'int:6'),
(u'REF',
'str:254'), (u'Z_ORDER', 'int:6')])}
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp')
Post by Ari Simmons
Post by Ari Simmons
c.schema
{'geometry': 'LineString', 'properties': OrderedDict([(u'ID',
'float:11'),
(u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE', 'str:254'),
(u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY', 'int:6'),
(u'REF',
'str:254'), (u'Z_ORDER', 'int:6'), (u'shield_type', 'str:254'),
(u'label',
'str:254'), (u'label_len', 'int:10'), (u'zoom', 'int:10')])}
meta = input.meta
# create new fields for the new schema
meta['schema']['properties']['shield_type'.encode("utf-8")] =
Post by Ari Simmons
Post by Ari Simmons
'str:254'
meta['schema']['properties']['label'.encode("utf-8")] =
'str:254'
meta['schema']['properties']['label_len'] = 'int:10'
meta['schema']['properties']['zoom'] = 'int:10'
n = item.copy()
new_data_attributes =
function_using_some_regex_parsing(item['properties']['REF']
...
Right now running this I just get an empty dictionary...I'm not sure
what
Post by Ari Simmons
Post by Ari Simmons
I'm missing...
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp')
Post by Ari Simmons
Post by Ari Simmons
c.schema
{'geometry': 'LineString', 'properties': OrderedDict()}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://lists.gispython.org/pipermail/community/attachments/20140528/2788d117/attachment.html
Post by Ari Simmons
Post by Ari Simmons
------------------------------
Message: 2
Date: Wed, 28 May 2014 22:29:39 -0600
From: Sean Gillies <sean.gillies at gmail.com>
Subject: Re: [Community] Trying to add fields using meta
To: "gispython.org community projects" <community at lists.gispython.org>
<
CAOodmJpBb34SBNkjkXT-CXEtNgNiH1zPb7jZeTcC1KWJ00hMCg at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi Ari,
Is there any chance that the
I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp
isn't
being written properly due to some logic in your function? My code
below
Post by Ari Simmons
Post by Ari Simmons
$ python
Python 2.7.6 (default, May 8 2014, 07:38:16)
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
import fiona
from collections import OrderedDict
OrderedDict([(u'ID',
'float:11'), (u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE',
'str:254'), (u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY',
'int:6'), (u'REF', 'str:254'), (u'Z_ORDER', 'int:6'), (u'shield_type',
'str:254'), (u'label', 'str:254'), (u'label_len', 'int:10'), (u'zoom',
'int:10')])}
with fiona.open('/tmp/foo.shp', 'w', crs={'init': 'epsg:4326'},
... pass
...
import subprocess
subprocess.check_output(['ogrinfo', '/tmp/foo.shp', 'foo'])
'INFO: Open of `/tmp/foo.shp\'\n using driver `ESRI Shapefile\'
0\nExtent: (0.000000, 0.000000) - (0.000000, 0.000000)\nLayer SRS
WKT:\nGEOGCS["GCS_WGS_1984",\n DATUM["WGS_1984",\n
SPHEROID["WGS_84",6378137,298.257223563]],\n
PRIMEM["Greenwich",0],\n
Post by Ari Simmons
Post by Ari Simmons
UNIT["Degree",0.017453292519943295]]\nID: Real (11.0)\nOSM_ID: Real
(19.0)\nNAME: String (254.0)\nTYPE: String (254.0)\nTUNNEL: Integer
(6.0)\nBRIDGE: Integer (6.0)\nONEWAY: Integer (6.0)\nREF: String
String
(254.0)\nlabel_len: Integer (10.0)\nzoom: Integer (10.0)\n'
On Wed, May 28, 2014 at 4:04 PM, Ari Simmons <ari.ucb.fire at gmail.com>
I am trying to copy the schema of an existing shapefile and add to it
in
an output shapefile. Somehow I am not getting anywhere with
this...though
https://pypi.python.org/pypi/Fiona) and it is just adding to a
dictionary.
For this object, I have this schema
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data.shp')
Post by Ari Simmons
Post by Ari Simmons
c.schema
{'geometry': 'LineString', 'properties': OrderedDict([(u'ID',
'float:11'),
(u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE', 'str:254'),
(u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY', 'int:6'),
(u'REF',
'str:254'), (u'Z_ORDER', 'int:6')])}
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp')
Post by Ari Simmons
Post by Ari Simmons
c.schema
{'geometry': 'LineString', 'properties': OrderedDict([(u'ID',
'float:11'),
(u'OSM_ID', 'float:19'), (u'NAME', 'str:254'), (u'TYPE', 'str:254'),
(u'TUNNEL', 'int:6'), (u'BRIDGE', 'int:6'), (u'ONEWAY', 'int:6'),
(u'REF',
'str:254'), (u'Z_ORDER', 'int:6'), (u'shield_type', 'str:254'),
(u'label',
'str:254'), (u'label_len', 'int:10'), (u'zoom', 'int:10')])}
meta = input.meta
# create new fields for the new schema
meta['schema']['properties']['shield_type'.encode("utf-8")] =
'str:254'
meta['schema']['properties']['label'.encode("utf-8")] =
'str:254'
meta['schema']['properties']['label_len'] = 'int:10'
meta['schema']['properties']['zoom'] = 'int:10'
n = item.copy()
new_data_attributes =
function_using_some_regex_parsing(item['properties']['REF']
...
Right now running this I just get an empty dictionary...I'm not sure
what
I'm missing...
c =
fiona.open(r'I:\It_24\115507_Road_Shields_Label_Processing\data\test_data_copy.shp')
Post by Ari Simmons
Post by Ari Simmons
c.schema
{'geometry': 'LineString', 'properties': OrderedDict()}
_______________________________________________
Community mailing list
Community at lists.gispython.org
http://lists.gispython.org/mailman/listinfo/community
--
Sean Gillies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://lists.gispython.org/pipermail/community/attachments/20140528/8908242a/attachment-0001.htm
Post by Ari Simmons
Post by Ari Simmons
------------------------------
_______________________________________________
Community mailing list
Community at lists.gispython.org
http://lists.gispython.org/mailman/listinfo/community
End of Community Digest, Vol 98, Issue 8
****************************************
_______________________________________________
Community mailing list
Community at lists.gispython.org
http://lists.gispython.org/mailman/listinfo/community
--
Sean Gillies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://lists.gispython.org/pipermail/community/attachments/20140601/2c0f2f9b/attachment.htm
------------------------------
_______________________________________________
Community mailing list
Community at lists.gispython.org
http://lists.gispython.org/mailman/listinfo/community
End of Community Digest, Vol 99, Issue 1
****************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gispython.org/pipermail/community/attachments/20140602/6bf99a31/attachment-0001.htm>
Sean Gillies
2014-06-02 20:36:48 UTC
Permalink
For best results, upgrade all the way up to 1.1.5.
Post by Ari Simmons
Hi Sean,
I think you are right....went down to 'shield_ty' and it worked. I am
running fiona 1.0.2..I can ugrade to 1.1.5 and test, or do you want me to
try 1.1.4 specifically?
-Ari
--
Sean Gillies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gispython.org/pipermail/community/attachments/20140602/3a41ac0d/attachment.htm>
Loading...