未加星标

PIG Loading CSV - Card Type Error

字体大小 | |
[前端(javascript) 所属分类 前端(javascript) | 发布者 店小二05 | 时间 2019 | 作者 红领巾 ] 0人收藏点击收藏

We aim to leverage PIG for largescale log analysis of our server logs. I need to load a PIG map datatype from a file.

I tried running a sample PIG script with the following data.

A line in my CSV file, named 'test' (to be processed by PIG) looks like,

151364,[ref#R813,highway#secondary]

My PIG Script

a = LOAD 'test' using PigStorage(',') AS (id:INT, m:MAP[]); DUMP a;

The idea is to load an int and the second element as a hashmap. However, when I dump, the int field get parsed correctly(and gets printed in the dump) but the map field is not parsed resulting in a parsing error.

Can someone please explain if I am missing something?

I think there is a delimiter related problem (such as field-delimiter is somehow effecting parsing of map field or it is confused with map-delimiter).

When this input data is used ( notice I used semicolon as field-delimiter ):

151364;[ref#R813,highway#secondary]

below is the output from my grunt shell:

grunt> a = LOAD '/tmp/temp2.txt' using PigStorage(';') AS (id:int, m:[]); grunt> dump a; ... (151364,[highway#secondary,ref#R813]) grunt> b = foreach a generate m#'ref'; grunt> dump b; (R813)

本文前端(javascript)相关术语:javascript是什么意思 javascript下载 javascript权威指南 javascript基础教程 javascript 正则表达式 javascript设计模式 javascript高级程序设计 精通javascript javascript教程

代码区博客精选文章
分页:12
转载请注明
本文标题:PIG Loading CSV - Card Type Error
本站链接:https://www.codesec.net/view/627814.html


1.凡CodeSecTeam转载的文章,均出自其它媒体或其他官网介绍,目的在于传递更多的信息,并不代表本站赞同其观点和其真实性负责;
2.转载的文章仅代表原创作者观点,与本站无关。其原创性以及文中陈述文字和内容未经本站证实,本站对该文以及其中全部或者部分内容、文字的真实性、完整性、及时性,不作出任何保证或承若;
3.如本站转载稿涉及版权等问题,请作者及时联系本站,我们会及时处理。
登录后可拥有收藏文章、关注作者等权限...
技术大类 技术大类 | 前端(javascript) | 评论(0) | 阅读(35)