Posted on

Combining CSV Files With Python

In my previous article, I shared splitting CSV files into small chunks. After you do some works on your splitted files, you may want to combine the result sets into one big file. Here you can find a small script for this purpose.

Python
  




xxxxxxxxxx
1
13


 
1 
import pandas as pd
2 
import csv
3 
Finded_URL = ['Finded_URL_1.csv', 'Finded_URL_501.csv','Finded_URL_1001.csv','Finded_URL_1501.csv','Finded_URL_2001.csv',
4 
'Finded_URL_2501.csv','Finded_URL_3001.csv','Finded_URL_3501.csv','Finded_URL_4001.csv','Finded_URL_4501.csv',
5 
'Finded_URL_5001.csv','Finded_URL_5501.csv','Finded_URL_6001.csv','Finded_URL_6501.csv','Finded_URL_7001.csv']
6 

          

          

           

            

             7
            

           

           
#combine all files in the list

          

          

           

            

             8
            

           

           
combined_csv = pd.concat([pd.read_csv(f,header=None) for f in Finded_URL])

          

          

           

            

             9
            

           

           

          

          

           

            

             10
            

           

           
combined_csv.head()

          

          

           

            

             11
            

           

           

          

          

           

            

             12
            

           

           
combined_csv.to_csv( "Finded_URL_All.csv", quotechar='"',

          

          

           

            

             13
            

           

           
          quoting=csv.QUOTE_ALL, index=False, encoding='utf-8')

          

         

        

       

      

     

     

      

     

     

      

       

      

     

    

   

  

 





	


	





		

		
Posted on 
Writing to a CSV File From Multiple Threads
	


			

			
							
		

	
	

		

 
I was writing document and metadata exporter that reads data from SharePoint and writes it to multiple files. I needed to boost up performance of my exporter and I went with multiple threads pumping out the data from SharePoint. One problem I faced — writing metadata to CSV files from multiple threads in parallel. This blog post shows how to do it using concurrent queues.

 

  
This posting uses CsvHelper library to write objects to CSV-files. Last time I covered this library in my blog post Generating CSV-files on .NET. 
	


	





		

		
Posted on 
How to Use the JMeterPluginsCMD Command Line
	


			

			
							
		

	
	

		
In this blog post, we are going to take a look at the JMeterPluginsCMD Command Line Tool. This command-line utility can generate PNG and CSV files based on JTL files. Thus, providing a convenient and quick way to extract the necessary data from JTL files in a presentable view, enabling easier analysis of Apache JMeter™ test results. The JMeterPluginsCMD Command Line Tool is very similar to the right-click context menu (Save Image as and Export to CSV options) on all graphs, and it's a better choice when you need to process a lot of JTL files or to generate a lot of PNG and CSV files based on one JTL file; it saves time.


There are two ways to install the JMeterPluginsCMS Command Line tool. The first method is to simply install the JMeter Plugins Manager, which includes the JMeterPluginsCMD. The second method is to download the archive from the official page and manually add the files to the corresponding folders of your JMeter. Don't forget to restart the application.