GuidesAn Introduction to File Handling in Go

An Introduction to File Handling in Go

Developer.com content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

Data can be stored in a file, which provides some form of a structure to the data and allows it to be stored permanently or temporarily depending upon the storage medium. Typically, main memory (or RAM) is used for temporary storage and hard disk, tape, and so on. are used for permanent storage. Programmatically handling a file essentially means working with it, such as getting metadata information, creating new files, or reading and writing data to and from a file format into a program’s internal data structure. All general-purpose programming languages provide these facilities through various libraries. In Go, the API for file handling is well-knitted into the standard architecture of the language and provides extensive support for it. Here we introduce the basic idea of file handling mechanisms and how to implement them in the Golang programming language.

Golang File Handling

Using a Go library, we can access the persistent storage of a computer or server and deal with the file system architecture of the underlying operating system. Dealing with filesystems is complex because modern operating systems provide file stores that map filenames to sequences of bytes and the representation may vary across various platforms. For example, porting file paths between Windows and Unix-like systems is tricky; one uses backslash and other slash characters as separators. Other systems may use different characters. The API design of the programming language must be consistent and provide a uniform interface to the programmer, irrespective of the differences in the file system structure of the underlying platform. This is just one example. There are many such issues. However, Go programmers, in most cases, do not have to deal with issues at such a low level (but we can actually manipulate paths in Go if need be). The library provides all that is required for file handling.

Read: Error Handling in Go.

File Handling Packages in Go

The built-in library of Go offers excellent support to perform file-related operations, such as creating a file, read/write operations, file rename or move, copy, getting metadata information of the file, and so forth. There are several packages involved in this process. Some key libraries and packages include:

The os package provides an API interface for file handling which is uniform across all operating systems. Although the design follows Unix standards, it provides a platform-independent interface to any operating system functionality. However, the error handling mechanism is unique to Go because, on failure, the return values are type errors rather than error numbers that we typically find in Unix systems. One particular advantage to that is that Go error types are loaded with information, which is particularly helpful in the debugging process. The os package provides functionality such as creation, deletion, opening a file, modification of permission, etc. The package therefore can be thought of as something that provides meta functionality related to the file handling process in Go.

The packages such as io, ioutil and bufio provide interfaces for I/O operations such as read and write operations on the content of the file.

  • The io package provides an interface for basic I/O primitives and wraps them into shared public interfaces that abstracts the functionality.
  • The ioutil package provides utility functions for I/O operations. Note that as of Go 1.16 the same functionalities can be accessed through io or os package directly.
  • The bufio provides an interface for buffered I/O operation with the file. Buffer is actually a temporary space in memory where data is stored and I/O operations are performed from this temporary space. It also means that if we are not using bufio we are basically having unbuffered I/O operations. Typically all I/O operations are unbuffered unless specified. The key advantage of having a buffer is that it minimizes system calls as well as disk I/O and is particularly suitable for block transfer of data. This is not suitable for single character-oriented I/O operations.

Some other packages that are often used or associated with file handling include fmt and log. The fmt package provides functionalities that help in formatting I/O with functions to read and write to the standard input and output. The log package provides the Logger type, which provides methods that help in formatting output.

Read: An Introduction to Database Programming in Go.

Constructing File Paths in Go

Deve;p[ers must use the right separator while constructing a file path. The filepath.Separator or the filepath.SeparatorString constants of the path/filepath package. The difference is that the former stores the character and the latter stores the string representation of the separator. It is safe to use string representation because the separator may be of multiple characters such as :: or // (although there is a very little chance of that). Here is a quick example of how to create a file path in Go:

func ManipulatePath() {
	dirs := []string{"home", "mano", "go-projects", "..", "src"}
	path := path.Join(dirs...)
	fmt.Printf("Path after join: %s\n", path)

	fmt.Printf("Path after split: ")
	splitted := filepath.SplitList(path)
	for _, d := range splitted {
		fmt.Printf("%s%c", d, filepath.Separator)
	}
}

Observe how the Join() and SplitList() functions combine a slice of strings into a single string with correct path separators and separate a string into its components, respectively.

How to Create Files in Go

Do you recall the touch command in Unix/Linux? It creates an empty file and if the file already exists it updates the access and modification time of the file. We can create a similar functionality with the help of the Create function in the os package. Here is an example. Note that it is not a full-fledged touch command but gives you the idea:

func CreateEmptyFile() {
	myFile, err := os.Create("sample.txt")
	if err != nil {
		log.Fatal("ERROR! ", err)
	}
	log.Println("Empty file created successfully. ", myFile)
	myFile.Close()
}

Getting File Information in Go

Sometimes we need to get meta-information about a specific file. The os.Stat function comes quite handy in extracting the info. Here is an example of how to use it:

func FileInfo() {
	fileInfo, err := os.Stat("sample.txt")
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println("File Name:", fileInfo.Name())
	fmt.Println("Size ", fileInfo.Size(), " bytes")
	fmt.Println("Permissions:", fileInfo.Mode())
	fmt.Println("Last modified:", fileInfo.ModTime())
	fmt.Println("Is Directory: ", fileInfo.IsDir())
}

Reading and Writing a File in Go

Reading from a file or writing into it is the basic operation performed on any file. However, reading and writing requirements may vary according to our needs. For example, we may want to read or write a block of bytes or a single character at a time. Here is an example illustrating how to read/write raw bytes of data to and from files in Go:

func FileRead(filePath string) {
	oFile, err := os.Open(filePath)
	if err != nil {
		log.Fatal(err)
	}
	defer oFile.Close()
	buff := make([]byte, 100)
	for no, err := oFile.Read(buff); err == nil; no, err = oFile.Read(buff) {
		if no > 0 {
			os.Stdout.Write(buff[0:no])
		}
	}
}

Note that files at the lowest level work on raw byte, so here we have used one. However, any convenient interface can be built on top of this abstraction. Observe how we have used the Write function to write to the Stdout file which is nothing but standard output. Since we are writing low-level bytes into the file a simple fmt.Println would print out the byte itself and not the character representation of the byte in this case. However, we can tweak fmt functions to get the desired outcome.

Now, armed with the idea, we can write something a bit more meaningful – something that does file copy: reading from one file and writing into another. Let’s try it with as simple a code as possible:

func Copy(src, dest string) {

	srcFile, err := os.Open(src)
	if err != nil {
		log.Fatal(err)
	}
	defer srcFile.Close()

	destFile, err := os.Create(dest)
	if err != nil {
		log.Fatal(err)
	}
	defer destFile.Close()

	numBytes, err := io.Copy(destFile, srcFile)
	if err != nil {
		log.Fatal(err)
	}
	log.Printf("Successfully copied %d bytes", numBytes)
	err = destFile.Sync()
	if err != nil {
		log.Fatal(err)
	}
}

Observe that, here, file read/write is happening under the garb of the io.Copy function. This function returns an error message, as well as the number of bytes, read and written into the destination file. Also, note that we have invoked the destFile.Sync() function. This function not only commits the current contents of the file to persistent storage but also flushes the file system’s in-memory copy of recently written data to the persistent store. Taking care of the memory footprints is always a good idea in programming.

Read: How to Use String in Go.

Go File Handling Tutorial

Understand that as we dive deeper, we cannot ignore file structure. After all, every file format provides some form of structure to the data. Now, files are the domain of the internal file system managed and maintained by the operating system. Different operating systems follow different file system architectures. As a result, things get more and more complex. Go API’s are excellent and crisp. A lot of magic can be done within a single statement. As a programmer, we may be relieved of the complexity but know that as we invoke the library functions a lot goes behind the scene. The closely-knit technologies as mentioned should be studied holistically to write good programs in file handling.

Get the Free Newsletter!

Subscribe to Developer Insider for top news, trends & analysis

Latest Posts

Related Stories