Skip to content

Blog

How to solve 'cannot find URL in scope' when accessing URL with XCode Playground ?

1. Purpose

In this post, I will demo how to solve the below exception when using xcode playground to test swift program that access a url.

The error in xcode is:

cannot find URL in scope
cannot find URLSession in scope

2. Environment

  • Mac OS
  • Xcode
  • Swift

3. The solution

3.1 The code that caused the problem

let url = URL(string: "http://www.stackoverflow.com")!
let task = URLSession.shared.dataTask(with: url) {(data, response, error) in
guard let data = data else { return }
print(String(data: data, encoding: .utf8)!)
}
task.resume()

When compiling, the xcode complains that:

Terminal window
cannot find URL in scope
cannot find URLSession in scope

3.2 The solution

Here is the solution, we should add some imports to the header of the code:

import Foundation
import PlaygroundSupport
PlaygroundPage.current.needsIndefiniteExecution = true

3.2.1 About swift import

An import declaration lets you access symbols that are declared outside the current file. The basic form imports the entire module; it consists of the import keyword followed by a module name:

import module

Providing more detail limits which symbols are imported—you can specify a specific submodule or a specific declaration within a module or submodule.

But how can I know which module the URL class belongs to ? We can just search this in Google:

swift class URL

Then we should see the link that points to apple developer documents, e.g. https://developer.apple.com/documentation/foundation/url

image-20210415200041850

3.2.2 About the needsIndefiniteExecution

We defined a variable needsIndefiniteExecution:

import PlaygroundSupport
PlaygroundPage.current.needsIndefiniteExecution = true

By default, when all top-level code is executed, the execution is terminated. When working with asynchronous code, enable indefinite execution to allow execution to continue after the end of the playground’s top-level code is reached. This, in turn, gives threads and callbacks time to execute.

We set this flag because we have an asynchronous process that would wait for the URL to be accessed and process the response, just as follows:

let task = URLSession.shared.dataTask(with: url) {(data, response, error) in
guard let data = data else { return }
print(String(data: data, encoding: .utf8)!)
}

In the above code, we passed a closure function to the dataTask function to process the URL response when the connection succeeded.

3.3 The total lines of code

Here is the whole code of our example:

import Foundation
import PlaygroundSupport
PlaygroundPage.current.needsIndefiniteExecution = true
let url = URL(string: "http://www.stackoverflow.com")!
let task = URLSession.shared.dataTask(with: url) {(data, response, error) in
guard let data = data else { return }
print(String(data: data, encoding: .utf8)!)
}
task.resume()

Now it works!

4. Summary

In this post, I demonstrated how to solve the ‘cannot find class in scope’ error in xcode playground, be sure to check you have imported the classes you are using .

How to copy file without overwriting destination file using python shutils?

1. Purpose

In this post, I will demonstrate how to copy file using python shutils , and will also show you how to copy without overwriting the destination file.

2. The Environment

  • Python 3
  • Shutils

3. The code

3.1 How to copy file?

This is the demo that shows how to copy file from src to dest:

copy_file_demo.py
import os,sys,shutil
def copy_file(src_path, src_file_name, dest_path, dest_file_name):
# construct the src path and file name
src_path_file_name = os.path.join(src_path, src_file_name)
# construct the dest path and file name
dest_path_file_name = os.path.join(dest_path, dest_file_name)
# do the real job
shutil.copyfile(src_path_file_name, dest_path_file_name)
print("copy from %s to %s ok" % (src_path_file_name,dest_path_file_name))
pass
if __name__ == '__main__':
src_path = sys.argv[1]
src_file_name = sys.argv[2]
dest_path = sys.argv[3]
dest_file_name = sys.argv[4]
copy_file(src_path,src_file_name,dest_path,dest_file_name)

3.2 Test the copy function

Suppose our working directory structure as follows:

.
└── working_directory/
├── copy_file_demo.py
├── logo.png
└── images/
└── readme.txt

Now I want to copy logo.png to images directory, and name it logo_bak.png, I do this job as follows:

Terminal window
$ python copy_file_demo.py . logo.png images logo_bak.png
copy from ./logo.png to images/logo_bak.png ok

After run the above command, I get this directory structure:

.
└── working_directory/
├── copy_file_demo.py
├── logo.png
└── images/
├── readme.txt
└── logo_bak.png

3.3 How to copy without overwritten

The code:

copy_file_demo.py
def copy_file_without_overwrite(src_path, src_file_name, dest_path, dest_file_name):
# construct the src path and file name
src_path_file_name = os.path.join(src_path, src_file_name)
# construct the dest path and file name
dest_path_file_name = os.path.join(dest_path, dest_file_name)
# test if the dest file exists, if false, do the copy, or else abort the copy operation.
if not os.path.exists(dest_path_file_name):
shutil.copyfile(src_path_file_name, dest_path_file_name)
print("copy from %s to %s ok" % (src_path_file_name, dest_path_file_name))
else:
print("already exist %s, copy aborted"%dest_path_file_name)
pass

3.4 Test the copy again

Our directory is :

.
└── working_directory/
├── copy_file_demo.py
├── logo.png
└── images/
├── readme.txt
└── logo_bak.png

Then I execute the below command in the working directory:

Terminal window
$ python copy_file_demo.py . logo.png images logo_bak.png
already exist images/logo.png, copy aborted

It works!

4. About the shutil

Shutil module in Python provides many functions of high-level operations on files and collections of files. … This module helps in automating process of copying and removal of files and directories. shutil. copy() method in Python is used to copy the content of source file to destination file or directory.

Shutil is the abbreviation of shell utility, which implements advanced functions such as file copying, moving, compression, and decompression in Python. It is a Python system module and does not require additional installation.

The commonly used functions in shutil are listed below:

  • shutil.copyfile(src, dst) Copy from source src to dst. Of course, the premise is that the target address has writable permissions. The exception information thrown is IOException. If the current dst already exists, it will be overwritten
  • shutil.move(src, dst) move file or rename
  • shutil.copymode(src, dst) just copy its permissions and other things will not be copied
  • shutil.copystat(src, dst) copy permission, last access time, last modification time
  • shutil.copy(src, dst) copy a file to a file or a directory
  • shutil.copy2(src, dst) is copied on the basis of copy and the last access time and modification time of the file are also copied, similar to cp -p
  • shutil.copy2(src, dst) If the file systems in the two locations are the same, it is equivalent to a rename operation, just rename; if it is not in the same file system, it is a move operation
  • shutil.copytree(olddir, newdir, True/Flase) copies olddir to newdir. If the third parameter is True, the symbolic link under the folder will be kept when copying the directory. If the third parameter is False, it will Generate a physical copy in the copied directory to replace the symbolic link
  • shutil.rmtree(src) recursively delete a directory and all contents in the directory

5. Summary

In this post, I demonstrated how to use shutil in python to copy files from here to there. And I also demonstrated how to avoid overwriting the dest file when copying. Thanks for your reading. Regards.

How to solve ValidationError(Deployment.spec): unknown field volumeClaimTemplates error?

Problem

When we install application in kubernetes(k8s), sometimes, we get the following error:

Terminal window
root@launch-advisor:~# kubectl apply -f deployment-redis.yaml
error: error validating "deployment-redis.yaml": error validating data: ValidationError(Deployment.spec): unknown field "volumeClaimTemplates" in io.k8s.api.apps.v1.DeploymentSpec; if you choose to ignore these errors, turn validation off with --validate=false

The content of deployment-redis.yaml is:

deployment-redis.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis
namespace: ns-bswen
labels:
app: redis
spec:
selector:
matchLabels:
app: redis
replicas: 1
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: redis:6.0.9
ports:
- containerPort: 6379
command: ["redis-server", "/etc/redis/redis.conf"]
volumeMounts:
- name: "redis-conf"
mountPath: /etc/redis/redis.conf
subPath: redis_conf
- name: "redis-data"
mountPath: "/var/lib/redis"
volumes:
- name: "redis-conf"
configMap:
name: "redis"
- name: "redis-data"
persistentVolumeClaim:
claimName: redis-pvc
volumeClaimTemplates:
- metadata:
name: redis-pvc
annotations:
volume.beta.kubernetes.io/storage-class: "my-nfs-storage"
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 100Mi

Notice that the above yaml is using the kubernetes resource type Deployment.

Environment

  • Docker: Server Version: 19.03.13

  • Kubectl version

    Terminal window
    Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.3", GitCommit:"06ad960bfd03b39c8310aaf92d1e7c12ce618213", GitTreeState:"clean", BuildDate:"2020-02-13T18:06:54Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"darwin/amd64"}
    Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:51:04Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

Reason

In Kubernetes (K8s), volumeClaimTemplates is a resource template used within StatefulSets to dynamically create persistent storage volumes (Persistent Volumes, PVs). VolumeClaimTemplates allow each Pod within a StatefulSet to have its own independent persistent storage, and PVCs (Persistent Volume Claims) corresponding to them can be automatically created through the template.

The role of volumeClaimTemplates:

  • Automate PVC creation: VolumeClaimTemplates enable you to automatically create PVCs for each Pod in a StatefulSet without manually configuring a PVC for each Pod. This is particularly useful for applications that require each Pod to have its own independent storage.
  • Ensure persistent storage: StatefulSets are primarily used for stateful applications (such as databases, distributed systems, etc.), which require ensuring that storage is not lost in cases of Pod restarts, migrations, etc. With volumeClaimTemplates, each Pod has its own PVC and mounted PV, ensuring data persistence.
  • Automatic binding of Persistent Volume (PV): Kubernetes automatically creates PVCs based on volumeClaimTemplates, and these PVCs are automatically bound to a suitable PV. This means that users do not need to manually manage the relationship between PVCs and PVs; Kubernetes automatically selects the appropriate PV based on storage resources and configured StorageClass.
  • Templated storage configuration: When using volumeClaimTemplates, you can define a template for PVCs (such as storage size, storage class, etc.), and Kubernetes will create corresponding PVCs for each Pod. This ensures that all Pods use the same storage configuration without the need to define storage details for each Pod individually.

But,

StatefulSet is useful for running things in cluster e.g Hadoop cluster, MySQL cluster, where each node has its own storage.

Solution

You can switch from Deployment to StatefulSet as follows:

deployment-redis.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: redis
namespace: ns-bswen
labels:
app: redis
spec:
serviceName: redis
selector:
matchLabels:
app: redis
replicas: 1
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: redis:6.0.9
ports:
- containerPort: 6379
command: ["redis-server", "/etc/redis/redis.conf"]
volumeMounts:
- name: "redis-conf"
mountPath: /etc/redis/redis.conf
subPath: redis_conf
- name: "redis-data"
mountPath: "/var/lib/redis"
volumes:
- name: "redis-conf"
configMap:
name: "redis"
- name: "redis-data"
persistentVolumeClaim:
claimName: redis-pvc
volumeClaimTemplates:
- metadata:
name: redis-pvc
annotations:
volume.beta.kubernetes.io/storage-class: "my-nfs-storage"
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 100Mi