Merge Two Lists in R

I have two lists first = list(a = 1, b = 2, c = 3) second = list(a = 2, b = 3, c = 4) I want to merge these two lists so the final product is $a [1] 1 2 $b [1] 2 3 $c [1] 3 4 Is there a...

Pandas column access w/column names containing spaces

If I import or create a pandas column that contains no spaces, I can access it as such: df1 = DataFrame({'key': ['b', 'b', 'a', 'c', 'a', 'a', 'b'], 'data1':...

How to add pandas data to an existing csv file?

I want to know if it is possible to use the pandas to_csv() function to add a dataframe to an existing csv file. The csv file has the same structure as the loaded data.

Get list from pandas DataFrame column headers

I want to get a list of the column headers from a pandas DataFrame. The DataFrame will come from user input so I won't know how many columns there will be or what they will be called. For...

Python pandas insert list into a cell

I have a list 'abc' and a dataframe 'df': abc = ['foo', 'bar'] df = A B 0 12 NaN 1 23 NaN I want to insert the list into cell 1B, so I want this result: A B 0 12 NaN 1 23 ...

How can a add a row to a data frame in R?

In R, how do you add a new row to a data frame once the data frame has already been initialized? So far I have this: df <- data.frame("hi", "bye") names(df) <- c("hello", "goodbye") #I am trying...

How to check if any value is NaN in a Pandas DataFrame

In Python Pandas, what's the best way to check whether a DataFrame has one (or more) NaN values? I know about the function pd.isnan, but this returns a DataFrame of booleans for each element. This...

Find the column name which has the maximum value for each row

I have a DataFrame like this one: In [7]: frame.head() Out[7]: Communications and...

move column in pandas dataframe

I have the following dataframe: a b x y 0 1 2 3 -1 1 2 4 6 -2 2 3 6 9 -3 3 4 8 12 -4 How can I move columns b and x such that they are the last 2 columns in the dataframe?...

Save Dataframe to csv directly to s3 Python

I have a pandas DataFrame that I want to upload to a new CSV file. The problem is that I don't want to save the file locally before transferring it to s3. Is there any method like to_csv for...

Sort a pandas dataframe based on DateTime field

I am trying to sort a dataframe based on DateTime field which is of datatype datetime64[ns]. My dataframe looks like this: Name DateTime1 P38 NaT P62 2016-07-13 16:03:32.771 P59 ...

How to transform Spark Dataframe columns to a single column of a string array

I want to know how can I "merge" multiple dataframe columns into one as a string array? For example, I have this dataframe: val df = sqlContext.createDataFrame(Seq((1, "Jack", "125", "Text"),...

Python: pandas merge multiple dataframes

I have diferent dataframes and need to merge them together based on the date column. If I only had two dataframes, I could use df1.merge(df2, on='date'), to do it with three dataframes, I use...

Pandas Explode lists with different lengths, into rows

I've got a pandas dataframe in which one of the columns contains lists with different lengths. The solutions to explode lists in pandas all assume that the lists to be exploded are all the same...

How to remove column and row labels from data frame?

I have a data frame Dim 1 Dim 2 Low Acceleration 0.615631233 0.000141725 Medium Acceleration 0.531160937 0.288548459 High Acceleration 0.001130792...

Using pandas Series.map to generate multiple columns for a DataFrame

Suppose I want to map a pandas Series to more than one column in a DataFrame using some function f(x). Ideally I would use one function per column. But suppose there is some heavy computation with...

Python Pandas update a dataframe value from another dataframe

I have two dataframes in python. I want to update rows in first dataframe using matching values from another dataframe. Second dataframe serves as an override. Here is an example with same data...

Python Pandas Assign a Dictionary Value to a Dataframe Column Based on Dictionary Key

NOT A DUPLICATE: The referenced answer maps the values in the dict to replace a column containing the a value equivalent to the key in the dict. My question is about adding the dict value to...

How can I subtract 3 hours from a datetime in a Pandas dataframe column?

I have two columns with datetime in gmt and I need subtract three hours from this datetime. For example in line 4 I need subtract startdate in 3 hours, the result it was: 08/02/2018 17:20:0. And...

How to unnest (explode) a column in a pandas DataFrame?

I have the following DataFrame where one of the columns is an object (list type cell): df=pd.DataFrame({'A':[1,2],'B':[[1,2],[1,2]]}) df Out[458]: A B 0 1 [1, 2] 1 2 [1, 2] My...

What is difference between DataFrame attribute and column

In [66]: data Out[66]: col1 col2 label 0 1.0 a c 1 2.0 b d 2 3.0 c e 3 0.0 d f 4 4.0 e 0 5 5.0 f 0 In [67]: data.label Out[67]: 0 ...

Manipulation of list of data frames in a for loop

I'm preparing data frames for analysis in R. I can prepare them separately correctly but I want to place the preparation in a for loop (or apply/lapply?) for obvious reasons. The initial code is...

Why does PANDAS only see one column to csv dataset with numerous columns?

I am new to and PANDAS and I am trying to work out why the shape of this csv dataset[https://www.kaggle.com/vfoufikos/airbnb-analysis-lisbon][1] is being shown as: (237, 1)? As it appears that the...

AttributeError: 'UnaryOp' object has no attribute 'evaluate' when using eval function in Python

for test_ind, case_data in test_df.iterrows(): case_data = case_data.to_frame().T rule = "Ask_before>-0.4843681 & 0.5255821<=BidVol_before<=0.07581073 & Volume>0.1107559" ...

Warning in xtfrm.data.frame(x) : cannot xtfrm data frames: error while ordering a dataframe numerically in R

I started learning R a couple of weeks ago, so I'm still very very new to R coding; I was trying to order a dataframe numerically, but on top of the ordered dataframe, resulted from the execution...

'Join columns must be present in data' error

I am having this weird issue. The following code works: Jakarta_Covid <- left_join(DKI_Jakarta, Covid_DF, by = c("Sub_District" = "Sub_District")) However the code...

How to add a checkbox in pandas dataframe

I have created a dataframe as: import pandas as pd data = [['Ankit'], ['Akshat' ]] df = pd.DataFrame(data, columns = ['Name']) Now, I want to insert a column PPA which has checkbox button as...

Pandas json column store as nested dataframe

I have data-frame which contains json column and is quiet huge and is not very efficient, i would like to store it as nested data frame. So sample data-frame looks like: id ...

Flatten a JSON file in Pandas

I am trying to change a JSON file to a pandas DataFrame. I have tried a number of solution such as pd.jason_normalize(data.json) but it doesn't work. The file appears to be more complex and has...

R function to extract top n scores from a dataframe and find their average using `apply` or dplyr `rowwise`

The dataframe looks like this df = data.frame(name = c("A","B","C"), exam1 = c(2,6,4), exam2 = c(3,5,6), exam3 = c(5,3,3), exam4 =...

Relevant tags